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ABSTRACT 


Sixteen alternative spaceborne nuclear power system concepts were ranked 
using multiattribute decision analysis. The purpose of the ranking was to 
identify promising concepts for further technology development and the issues 
associated with such development. 

Eleven individuals representing four groups were successfully interviewed 
to obtain their preferences. The four groups were: safety, systems definition 

and design, technology assessment, and mission analysis. 

The ranking results were consistent from group to group and for different 
utility function models for individuals. The highest ranked systems were the 
heat-pipe thermoelectric systems, heat-pipe Stirling, in-core thermionic, and 
liquid-metal thermoelectric systems. The next group contained the liquid-metal 
Stirling, heat-pipe AMTEC (Alkali Metal Thermoelectric Converter), heat-pipe 
Bray ton, liquid-metal out-of-core thermionic, and heat-pipe Rankine systems. 

The least preferred systems were the liquid-metal AMTEC, heat-pipe thermo- 
photovoltaic, liquid-metal Bray ton and Rankine, and gas-cooled Bray ton. 

Although the R&D community subsequently discounted the heat-pipe reactor 
systems, the three non-heat-pipe technologies selected matched the top 
three non-heat-pipe systems ranked by this study (liquid-metal thermo- 
electric, in-core thermionic, and liquid-metal Stirling). 

The multiattribute decision analysis process was viewed as a useful 
exercise for identifying options which needed further development. The analy- 
sis highlighted the need for additional and higher quality technical data as 
well as a need to provide an on-line capability to display source calculations 
interactively* An approach was suggested for displaying such traceability. 


iii 


FOREWORD 


The Defense Advanced Projects Research Agency, together with the 
U.S. Department of Energy and NASA, established the Space Power-100 Devel- 
opment Project to assess the potential and demonstrate the feasibility of 
developing a nuclear power system for operation in space. The SP-100 R&D 
Project Office was given the responsibility to assess the state of the 
required technologies and make recommendations for research in support of 
such a development from a systems perspective. Therefore, the objectives 
of the assessment were to characterize and give priority to the various 
subsystem technologies and system concepts through the use of simulation, 
based on projections of the subsystem capabilities. 

This report describes the multiattribute decision analysis that ranked 
16 power system concepts using the preferences of 11 individuals, all knowl- 
edgeable in advanced nuclear reactor and power-conversion technologies* The 
advanced system concepts were designed to meet a 100-kW power requirement, 
3000-kg mass requirement, and 7-year lifetime. 

The report is divided into two volumes. Volume I is a summary of the 
multiattribute decision analysis. Volume II describes the multiattribute 
decision analysis and provides detailed technical information on the 
methodology and system concepts. 


IV 



ACKNOWLEDGMENTS 


This work could not have been completed without the support of many 
individuals who gave generously of their time. At the Jet Propulsion 
Laboratory, the authors wish to thank Vince Truscello, Jim French, Jack Mondt, 
and Tosh Fujita. Considerable thanks are due to those individuals who were 
interviewed as to their preferences for the attributes of the systems that 
were evaluated. Those organizations whose representatives participated are: 


Organization 


Location 


Air Force Weapons Laboratory 
Jet Propulsion Laboratory 
Los Alamos National Laboratories 
NASA Lewis Research Center 


Albuquerque, New Mexico 
Pasadena, California 
Albuquerque, New Mexico 
Cleveland, Ohio 


A very special thanks is owed to Fran Mulvehill who patiently and cheerfully 
typed this manuscript with its many difficult tables. Notwithstanding the 
help of these individuals above, the responsibility for this report rests with 
the authors. 


This work was conducted by the Jet Propulsion Laboratory's SP-100 
Project, which is a tri-agency agreement under the Defense Advanced Projects 
Research Agency, ILS. Department of Energy, and NASA. 


v 


CONTENTS 


I. INTRODUCTION 

A. INTRODUCTION 

B. INTERVIEWS 

C. RANKINGS 

D. CONCORDANCE AMONG RANKINGS 1-4 

E. REPORT 

II. METHODOLOGY 

A. MULTIATTRIBUTE DECISION ANALYSIS 2-1 

B. RISK ANALYSIS 2-13 

C. CONCORDANCE 2-14 

III. OBJECTIVES, CRITERIA, AND ATTRIBUTES . 3-1 

A. INTRODUCTION 

B. HIERARCHY OF OBJECTIVES, CRITERIA, AND ATTRIBUTES 3-1 

C. OBJECTIVES FOR ASSESSING SYSTEM CONCEPT ALTERNATIVES .... 3-2 

D. OBJECTIVES, CRITERIA, AND ATTRIBUTE SETS 3-4 

E. DISCUSSION OF ATTRIBUTES 

F. DETERMINATION OF ATTRIBUTE SCALES 3-7 

IV. ALTERNATIVES AND STATE DATA 4-1 

A. INTRODUCTION 

B. ALTERNATIVE SYSTEM CONCEPTS 

C. SYSTEM CONCEPT ATTRIBUTE STATE DATA ..... 4-2 


vii 


V. INTERVIEWS 5-1 

A. INTRODUCTION 5-1 

B. INTERVIEWEES 5-1 

C. INTERVIEW PROCESS 5-2 

D. SAMPLE QUESTIONS 5-4 

E. INTERVIEW PROCESS REFERENCES 5-4 

F. INTERVIEW RESULTS 5-4 

VI. RANKING ANALYSIS AND DISCUSSION 6-1 

A. OVERVIEW OF THE DECISION ANALYSIS RESULTS 6-1 

B. MATEUS COMPUTER RUNS 6-1 

C. MULTIATTRIBUTE RESULTS FOR NOMINAL ATTRIBUTE DATA 6-3 

D. RESULTS OF VARIATIONS ON MULTIATTRIBUTE NOMINAL DATA .... 6-3 

E. GENERAL CONCLUSIONS 6-12 

F. CRITIQUE OF MULTIATTRIBUTE DECISION ANALYSIS 

METHODOLOGY 6-12 

VII. CONCORDANCE OF RANKINGS 7-1 

A. INTRODUCTION 7-1 

VIII. REFERENCES 8-1 

Figures 

1- 1. Ranking Methodology Flow Diagram 1-2 

2- 1. Relationship between System Value Models 2-3 

2-2. Typical Decision Tree 2-5 

2-3. Hierarchy of Objectives, Criteria, and Attributes 2-7 

2-4. Examples of Increasing Utility Functions for 

Different Risk Attitudes 2-15 


viii 



3-1* Hierarchy of Objectives, Subobjectives, Criteria, and 


Attributes for Spaceborne Power System Concepts ..•••• 3-6 

5-1. Decision-Analysis Interview Flow Chart 5-3 

5-2. Sample Interview Question, Radiator Area 5-5 

5-3. Sample of Interview Question, Order of 

Attribute Importance 5-6 

5-4. Sample of Interview Question, Importance of 

Radiator Area ..... 5-7 


Tables 

2-1. Table of Critical Values of M S n in the Kendall 


Coefficient of Concordance 2-16 

3-1. Requirements for Space Power Concepts 3-4 

3- 2. Attributes with Ranges for Cost and Performance 3-8 

4- 1. Alternative System Concepts with Abbreviations 4-1 

4-2. Attribute State Data for the Sixteen Alternative 

System Concepts 4-3 

4- 3. Attributes within Range x 10% of Most 

Preferred State 4-4 

5- 1. Preference Data for All Interviews 5-8 

5-2. Preference Data For Interviews by Group 

(a) Safety Area 5-9 

(b) System Definition and Design Area 5-10 

(c) Technology Assessment Area 5-11 

5-3. Preference Data from Interviews, Importance of 

Attributes 5-12 

5- 4. Ranking of Attribute Importance 5-13 

6- 1. Rankings for All Individuals for Set 1 6-4 

6-2. Multiattribute Decision Analysis : Summary of Advanced 

Vehicle Rankings Using Nominal States, Multiplicative 
Model, 5-pt Utilities 6-5 

6-3. Rankings by Each of the Eight Attributes 6-6 


ix 


6-4(a) Multiattribute Decision Analysis: Summary of System 

Concept Rankings; Results for Safety Group 6-7 

6-4(b) Multiattribute Decision Analysis: Summary of 

System Concept Rankings; Results for Systems 

Definition and Design Group 6-8 

6- 4(c) Multiattribute Decision Analysis: Summary of 

System Concept Rankings; Results for Technology 

Assessment Group 6-9 

7- 1. Summary of Kendall's Coefficient of Concordance (W) and 

the Associated Chi-Square Values (X^) for all Runs: 

Individuals Within Groups 7-2 

7-2. Summary of Kendall's Coefficient of Concordance (W) and 
the Associated Chi-Square Values (X^) for all Runs: 

Group Decision Rules 7-3 

APPENDICES 

A. QUESTIONNAIRE USED FOR INTERVIEWS A-l 

B. COMPUTER PROGRAMS USED B-l 


x 



SECTION I 


INTRODUCTION AND SUMMARY 


A. INTRODUCTION 

Sixteen alternative spaceborne nuclear power concepts were studied and 
ranked using a multiattribute decision analysis. The system concepts were all 
designed to meet a 100-kW power level and 3000-kg mass limit and to operate in 
the space environment for a 7-year lifetime. The systems included seven heat- 
pipe cooled and seven liquid-metal cooled systems with a variety of dynamic 
and static power conversion systems. One gas-cooled system and an in-core 
system were also examined. The conversion systems included Brayton, Stirling, 
Rankine, thermoelectric, thermophotovoltaic, thermionic, and AMTEC technolo- 
gies. 


Ten attributes were intended to be used in the ranking, but two were 
not included because it was believed they would not have affected the rankings 
significantly — estimated development cost and production cost in 1983 dollars. 
Thus, only eight attributes impacted the rankings: safety, radiator area, 

design reliability, technical maturity, estimated cost to reach technical 
feasibility, survivability, dormancy capability, and producibility. 

The methodology used to rank the system concepts was multiattribute 
decision analysis with the base case model using a multiplicative multi- 
attribute utility function (Reference 1). A linear multiattribute utility 
function was also used to compare with rankings derived from the base case 
model. The methodology combines an individual's preferences with analytical 
estimates of the attribute states to produce a ranking for that individual. 

A flow diagram for the method is shown in Figure 1-1. 

Because several individuals are involved in a major decision such as the 
ranking of technical concepts, the rankings had to be determined for groups as 
well as for individuals. Thus, the Methodology Section includes discussion of 
group-decision rules. Three group-decision rules were used to aggregate indi- 
vidual rankings because there is no definitive rule for groups: rank sum rule, 

additive utility rule, and Nash bargaining rule. 


B. INTERVIEWS 

Eleven individuals, knowledgeable in spaceborne power system technolo- 
gies, were successfully interviewed to obtain their preferences with regard 
to the eight attributes selected. These individuals were drawn from organi- 
zations with: 

(1) Ongoing research and development programs in advanced power 
conversion systems. 

(2) A proven record of achievement in the research and development of 
nuclear power systems. 




Figure 1-1. Ranking Methodology Flow Diagram 


1-2 






(3) An understanding of space environment issues which have direct 
impact on developing nuclear power technologies for space 
applications. 

These individuals represented four distinct groups: 

(1) Safety Group, This group was concerned with a range of safety 
issues from ground development through launch, on-orbit operation, 
and re-entry. 

(2) Systems Definition and Design Group. This group was concerned 
with the design issues and options involved in the development 
and deployment of the technology, 

(3) Technology Assessment Working Group. This group was involved in 
assessing the technical issues facing the demonstration of tech- 
nical feasibility for such power systems. 

(4) Mission Analysis Group. This area involved the concerns of 
possible mission users who would utilize the system concepts. 


C. RANKINGS 

Rankings were calculated for the 11 individuals successfully interviewed 
and for the four groups that they represented. Rankings for the individuals 
were calculated using several different multiattribute utility models and with 
each of the attributes removed. Rankings for the groups were calculated using 
three different group decision rules. 

The ranking results were quite consistent from group to group and for 
different utility function models for individuals. Generally, the rankings 
fell into four areas: most preferred concepts (those high-ranking systems 

whose rankings were unchanged by various assumptions about the multiattribute 
decision model), preferred (those systems whose (high) rankings varied with 
changes in the multiattribute decision model assumptions but remained clus- 
tered together near the high end of the rankings), intermediate (those systems 
whose rankings varied with changes in the multiattribute decision model assump- 
tions but remained clustered together near the low end of the rankings), and 
least preferred (those low-ranking systems whose rankings were virtually 
unchanged by various assumptions about the multiattribute decision model). 

The most preferred systems were the heat-pipe thermoelectrics (HTEP, HTEPa). 

The preferred systems were the heat-pipe Stirling (HSH), in-core thermionic 
(ICT), and liquid-metal thermoelectric (LTEP). The intermediate systems 
were the liquid-metal Stirling (LSH), heat-pipe AMTEC (HAP), heat-pipe 
Brayton (HBO) , liquid-metal out-of-core thermionic (LOCTP), and heat-pipe 
Rankine (HRL). The least preferred concepts were the liquid-metal AMTEC 
(LAP), heat-pipe thermophotovoltaic (HTPVP) , liquid-metal Brayton (LBO), 
liquid-metal Rankine (LRL), and gas-cooled Brayton (GBH) . 

The above rankings were used to initiate planning for the technical 
development of promising options within the project time frame. In par- 
ticular, the rankings were used to identify technology areas for more 
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comprehensive research. A subsequent technology downscoping evaluation elimi- 
nated almost all of the heat-pipe concepts as being riskier than previously 
thought with a limited operational database. The results of the present 
analysis had a direct impact on the list of systems which were candidates 
for this downscoping effort. It should be noted that the rank ordering of 
the remaining systems (after removing the heat-pipe systems) was substan- 
tially the same with the preliminary results obtained herein. 


D. CONCORDANCE AMONG RANKINGS 

The concordance or agreement among the rankings was calculated for 
individuals within groups, different group decision rules, and different 
multiattribute utility models. The concordance calculations were carried 
out to ascertain how robust the rankings were. In general, the rankings 
were highly concordant across individuals, different group decision rules, 
and different raultiat tribute utility models, implying that the rankings were 
indeed robust. 

The robustness of the rankings was due to (1) a general consensus 
regarding the importance of the safety and technical maturity attributes; 
and (2) the dominance of the system data in pre-determining the high and 
low end rankings. 


E. REPORT 

This Volume consists of seven sections: an introduction (Section I); 

methodology (Section II); description of the attributes (Section III); listing 
of the alternatives and state data (Section IV); summary of the interviews and 
preference data (Section V); presentation and analysis of the rankings and 
results (Section VI); and summary of the concordance of rankings (Section VII). 
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SECTION II 


METHODOLOGY 


This section describes and illustrates the methodology used to evaluate 
and compare alternative spaceborne power concepts. The methodology consists 
of a number of steps which, in short, characterize the alternative approaches 
under different design options and operating environments, assign utility 
values to the alternatives, and rank the alternatives based on these utili- 
ties. Tests of concordance of the rankings for different individuals, groups, 
methodologies, and attribute sets were carried out and are discussed in 
Section VII. 

The evaluation methodology may be summarized as follows. The process 
begins with the selection of a set of descriptive but quantifiable attributes 
designed to characterize each system. Values for this set of attributes are 
then generated for each alternate approach that specify its response (e.g., 
performance or cost) under different design options and operating environments. 
(The attributes are discussed in Section III.) A decision tree can be con- 
structed to relate economic, technological, and environmental uncertainties 
(i.e., the operating environment) to the cost and performance outcomes (i.e., 
attribute values) of the alternative power concepts. Multiattribute utility 
functions that reflect the preferences and perceptions of knowledgeable indi- 
viduals are generated, based on interviews with selected personnel. The func- 
tions are then employed to generate a multiattribute utility value for each 
system, based on its characteristics under the scenarios reflected within the 
decision tree. The decision tree is used to compute an expected multiattribute 
utility value for each alternative, the expected value being taken over the 
scenario probability distribution. Alternative systems are ranked according 
to this expected multiattribute utility value. 


A. MULTIATTRIBUTE DECISION ANALYSIS 
1. Overview 

Multiattribute decision analysis is a methodology for providing 
information to decision-makers for comparing and selecting from among complex 
alternative systems in the presence of uncertainty. The methodology of mul- 
tiattribute decision analysis is derived from the techniques of operations 
research, statistics, economics, mathematics, and psychology. Thus, research- 
ers from a wide range of disciplines have participated in the development of 
raultiattribute decision analysis. The first books and papers on the subject 
appeared in the late 1960s (References 2 through 5). The most practical, 
extensive, and complete presentation of an approach to multiattribute deci- 
sion analysis is given in the 1976 work of Keeney and Raiffa (see Reference 1). 
Although several approaches to multiattribute decision analysis have been 
developed (References 6 through 19), the method used in this report corresponds 
to an abbreviated form of that of Keeney and Raiffa. A brief introduction to 
multiattribute decision analysis, discussing primarily the Keeney and Raiffa 
methodology, is given in Feinberg and Miles (Reference 20). The assumptions 
needed for the abbreviated form used here are discussed at the end of sub- 
section A-4. 
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Every systems analysis involving the preference ranking of alternative 
systems, whatever the specific methodology, requires two kinds of models. One 
is a "system model" and is representative of the alternative systems (including 
any uncertainties) under consideration. The other is a "value model" and is 
representative of the preference structure of the decision-makers whose prefer- 
ences are being assessed. 

The system model describes the alternative systems available to the 
decision-makers in terms of the risk and possible outcomes that could result 
from each system. Risk arises from the technological and economic uncertainty 
associated with each alternative system and from the uncertain environment in 
which the systems would be required to perform. The outcomes describe the 
possible consequences of the alternative systems. Because of the element of 
risk, the selection of a specific system does not in general guarantee a 
specific outcome, but rather results in a probabilistic situation in which 
only one of several outcomes may occur. These outcomes, with their measurable 
attributes, then form the input to the value model. The value model assesses 
the outcomes in terms of the preferences of the decision-makers for the various 
outcomes. The measurable attributes of the outcomes are aggregated algebrai- 
cally in a formula (called a multiattribute utility function) whose functional 
form and parameters are determined by the preference structure of the decision- 
makers. The output of the value model is a multiattribute utility function 
value for each outcome (outcome utility). These outcome utilities are entered 
back into the system model where an alternative system utility can be calcu- 
lated for each alternative system simply by taking the expected utility value 
of the outcomes associated with each alternative system. These alternative 
system utilities then define a preference ranking over the alternative sys- 
tems, with greater alternative system utilities being more preferred. 

The relationship between the system model and the value model is illus- 
trated in Figure 2-1, which shows that the combination of a selected system and 
a realized state of uncertainty results in the output from the system model to 
the value model of a specific outcome. The output of the value model is an 
outcome utility. The probabilistic combination of the outcome utilities of the 
outcomes associated with a specific alternative system determine an alternative 
system utility in the system model. Comparison of the alternative system util- 
ities for all the alternative systems under consideration results in an alter- 
native system ranking as the output from the system model. 


2. Decision Trees 

Decision trees are used to represent the system model and the 
inputs to the system model at the gross level required for the decision 
analysis. Decision trees are graphically depicted by decision nodes (repre- 
sented by squares), with alternative paths emanating from them; and by chance 
nodes (represented by circles), with probabilistic paths emanating from them. 
All paths either terminate at another node or terminate at an outcome, which 
is a description of the consequence of traversing a specific set of paths and 
nodes through the decison tree from beginning to end. There can be only one 
originating node (either a decision node or a chance node). There can be many 
outcomes terminating the decision tree, depending on the complexity of the 
decision tree. 
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UNCERTAINTY 



Figure 2-1. Relationship Between System Value Models 
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Figure 2-2 shows a typical decision tree, terminating in 10 outcomes. 

The symbols "D^" stand for the ith decision node ("D" for decision). The 
symbols "Pj" stand for the j th chance node ("P" for probabilistic). The 
symbols stand for the kth outcome ("C" for consequence). Every path 

emanating from a decision node corresponds to an alternative that the deci- 
sionmakers can select, where "A ^ 11 stands for the £th alternative selected 
at the ith decision node. The decision-makers can select one and only one 
path at each decision node. Every path Pj m emanating from a chance node 
corresponds to one of the uncertain and uncontrollable chance states that 
can occur at that node, where p: m is the probability that the mth chance 
state will be realized at the j th chance node. The Pj m s must obey the laws 
of probability theory. Thus, one and only one chance path can be realized 
from a chance node, and the Pj m s must sum to 1.0. 

The chance nodes and their associated chance paths and probabilities are 
called "gambles" or "lotteries" in the literature. This report shall refer to 
them as gambles. An example of a gamble would be a flip of a coin, which 
could be expected to come up heads 50% of the time and tails 50% of the time. 
Graphically, such a gamble would be displayed as: 



HEADS 


TAILS 


Figure 2-2 has an example of every kind of node-path-outcome relation- 
ship. There are examples of decision-node to decision-node paths, decision- 
node to chance-node paths, decision-node to outcome paths, chance-node to 
decision-node paths, chance-node to chance-node paths, and chance-node to 
outcome paths. 

As an example of how the decision tree might be traversed, imagine that 
the decision-maker selects Alternative Path k\2 at Decision Node D]_, where 
he must start. This leads to Chance Node P^ where Chance Path Pj^ is rea- 
lized, leading to Chance Node P 3 , where Chance Path P 32 is realized, and 
terminates with Outcome C^q. 


3. Objectives Hierarchy 

The outcomes that terminate the decision tree are to be described 
in terms of an objectives hierarchy that ( 1 ) expresses the preference structure 
of the decision-makers, and ( 2 ) is constructed in a manner compatible with the 
quantification and mathematical conditions required by a multiattribute utility 
function of the value model. The objectives hierarchy expresses the preference 
structure of the decision-makers in ever increasing detail as one proceeds down 
through the hierarchy from overall objective to a lower-level hierarchy of sub- 
objectives. Below the subobjectives are "criteria." The criteria must permit 
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Figure 2-2. Typical Decision Tree 












the quantification of performance of the alternatives with respect to the sub- 
objectives* Associated with each criterion is an "attribute, 11 a quantity that 
can be measured and for which the decision-makers can express preferences for 
its various states. Figure 2-3 shows an objectives hierarchy with the associ- 
ated attributes. 

The set of attributes must satisfy the following requirements for the 
value model to be a valid representative of the preference structure of the 
decision-makers: 

(1) Completeness : The set of attributes should characterize all of 

the factors to be considered in the decision-making process* 

(2) Comprehensiveness : Each attribute should adequately characterize 

its associated criterion. 

(3) Importance : Each attribute should represent a significant crite- 

rion in the decision-making process, at least in the sense that the 
attribute has the potential for affecting the preference ordering 
of the alternatives under consideration. 

(4) Measurability : Each attribute should be capable of being objec- 

tively or subjectively quantified; technically, this requires that 
it be possible to establish an attribute utility function for each 
attribute. 

(5) Familiarity : Each attribute should be understandable to the 
decision-makers in the sense that they should be able to identify 
preferences for different states of the attribute for gambles over 
the states of the attribute. 

(6) Nonredundancy : Two attributes should not measure the same 

criterion, thus resulting in double counting. 

(7) Independence : The value model should be so structured that 

changes within certain limits in the state of one attribute should 
not affect the preference ordering for states of another attribute 
or the preference ordering for gambles over the states of another 
attribute. 

4. Attribute Utility Functions and the Multiattribute Utility Function 

The set of attributes associated with the objectives* hierarchy 
must satisfy the aforementioned measurability and mathematical requirements. 

If it satisfies these requirements, then it is possible to formulate a mathe- 
matical function (called a multiattribute utility function) that will assign 
numbers (called outcome utilities) to the set of attribute states characteriz- 
ing an outcome. The multiattribute utility function that was used is that 
of Keeney and Raiffa (Reference 1). The outcome utilities generated by the 
Keeney and Raiffa multiattribute utility function have the properties of 
Von Neumann and Morgenstern utilities (Reference 23), that is: 
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( 1 ) 


Greater outcome utility values correspond to more preferred 
outcomes. 

(2) The utility value to be assigned to a gamble is the expected 
value of the outcome utilities of the gamble. 

The mathematical axioms that must be valid for these two properties to 
hold were first derived by Von Neumann and Morgenstern (see Reference 23). 
Elementary expositions of these axioms are given in Hadley (Reference 24) and 
Luce and Raiffa (Reference 25). An intermediate exposition is given in 
DeGroot (Reference 26). An advanced exposition is given in Fishburn 
(Reference 27). 

To every outcome "C," an N-diraensional vector of attributes x = (x^, . x^) 
will be associated, the set of which satisfy the attribute requirements pre- 
sented in the preceding subsection. Most of the attribute requirements are 
self-evident. The seventh requirement, that of attribute independence, is a 
condition that makes it possible to consider preferences between states of a 
specific attribute, without consideration of the states of the other N-l attri- 
butes. It is thus possible to construct an attribute utility function that is 
independent of the other attribute states, and which, like the outcome utility 
function, satisfies the Von Neumann and Morgenstern properties for utility 
functions. This condition of independence, or some equivalent mathematical 
condition (see Reference 1 for alternative formulations), is necessary for the 
Keeney and Raiffa methodology. It is necessary to verify that this condition 
is valid in practice, or more correctly, to test and identify the bounds of 
its validity. 

To continue the discussion from this point on, it is necessary to 
introduce some mathematical notation: 

x n = The state of the nth attribute. 

xg = The least-preferred state to be considered for the nth 
attribute. 

x£ = The most-preferred state to be considered for the nth 
attribute. 

x = The vector (x^, x^) of attribute states 

characterizing a specific outcome. 

x° = An outcome constructed from the least preferred states of all 
the attributes. x° = (x°, ..., x°) . 

x = An outcome constructed from the most preferred states of all 
attributes, x* = (xf, ..., x^). 

( x n y xg) = An outcome in which all attributes except the nth attribute 
are at their least-preferred state. 
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u n (x n ) = The attribute utility of the nth attribute. 
u(x) - The outcome utility of the outcome x. 


The attribute scaling constant for the nth attribute. 
k r - 


v n 


u ^ x n * x n ^ • 


k - The master scaling constant for the multiattribute utility 
equation. It is an algebraic function of the k n s. 

With this mathematical notation, the discussion can proceed to how 
attribute utility functions and the attribute scaling functions are assessed. 
The mathematics permit the arbitrary assignments: 


u n (x°) - 0.0 


and 


u n (x*) -1.0 

Thus, the attribute utility function values will range from 0.0 to 1.0. 
Attribute utility function values for attribute states x n intermediate 
between x£ and x n are assessed by determining a value of p n such that 
the decision makers or their designated experts are indifferent between 
receiving x n for sure or a gamble that yields xg with probability p n or 
x* with probability l-p n . Graphically, assess p n , so that: 



where " ~ " means indifference. 

It follows from the mathematics that: 


u n^ x n) 58 Pn 

This indifference relation is repeated for various attribute states until 
either a continuous utility function can be approximated or enough discrete 
points have been assessed for the attribute states under consideration in the 
analysis. 

A similar approach is used to assess the scaling constants k n . A value 
for k n is assessed such that the following indifference relationship holds: 
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With this assessed information, the multiattribute utility equation can 
be solved to yield an outcome utility value for any outcome under considera- 
tion. The multiattribute utility function can now be stated: 


If 


N 

£ 


n=l 


k f 1.0 
n 


then 



where the master scaling constant k is solved for from the equation: 


1 + k 


N 

TT (ink) 

n 


If 


N 


£ 


k 

n 


1.0 


then there is an additive utility function, 

N 

u(x) = V k u (x ) 

" n n n 
n=l 

The outcome utility function values, like the attribute utility function 
values, will all range from 0.0 to 1.0 with u(x°) = 0.0 and u(x*) = 1.0. 
Although the mathematical equations appear complex, they can be easily solved, 
and the information required in the interviews with the decision-makers can be 
minimized. An extended discussion of these equations, their solution, and the 
assessment of the required data, together with examples taken from actual 
applications, is given in Keeney and Raiffa (see Reference 1). 
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In this study, an abbreviated form of Keeney and Raiffa' s methodology 
was used to reduce the interview time for the interviewee. An assumption was 
made that utility independence of each attribute implies pair-wise utility 
independence (i.e., the attributes exhibit utility independence when taken two 
at a time). This assumption allows the use of Formulation (4) of Theorem 6.2 
of Keeney and Raiffa (see Reference 1). Given single-attribute utility 
independence, the authors could not construct a realistic example where 
pair-wise utility independence would be violated. 

The abbreviated form satisfies the multilinear model shown in Theorem 
6.3 of Keeney and Raiffa. However, the multilinear form requires the 
assessment of 2 n -2 scaling constants, where n is the number of attributes. 

With n - 8 attributes, 254 scaling constants would be needed, requiring 
extensive time for both the interviewer and interviewee. 


5. Ranking the Alternative Systems 

The steps needed prior to ranking the alternatives are: the 

development of a decision tree, the determination of probabilities for the 
decision of an objectives hierarchy, the quantification of the criteria in 
terms of measurable attributes, and the determination of a raultiattribute 
utility function with attribute utility functions and attribute scaling 
constants corresponding to the preference structure of the decision makers. 

The ranking of the alternative systems proceeds as follows (see Figure 2-2): 

(1) Use the multiattribute utility function to calculate outcome 
utilities for all of the outcomes of the decision tree. 

(2) Calculate a utility value to be assigned to all chance nodes 
by taking the expected utility value of the utilities 
assigned to the termination of the chance paths of the 
chance nodes. The chance paths may terminate at outcomes, 
other chance nodes, decision nodes, or a combination of 
these. 

(3) Calculate a utility value for all decision nodes by 
selecting the decision path that terminates in an outcome, 
chance node, or decision node with the highest utility 
value. The utility value of that path shall be the utility 
value assigned to the decision node. 

The decision tree for this study has an originating decision node whose 
decision paths correspond to the alternative systems under consideration. 

Steps (1) through (3) are performed by starting with the outcomes as shown in 
Figure 2-2 and assigning utility values to these outcomes. Then Steps (2) and 
(3) are performed by a "folding back 1 ' process, proceeding from right to left, 
and assigning utility values to the chance nodes and the decision nodes. 
Finally, utility values are assigned to the decision paths emanating from the 
orginating decision node on the left. These utility values are the ones 
assigned to the alternative systems. Because greater utility values correspond 
to more preferred systems, a rank order in preference for the alternative 
system can be assigned in correspondence with the utility values. A quantifi- 
able and tangible measure of the strength of preference between the alternative 
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systems can be obtained by referencing each alternative system to a set of 
systems where only one attribute, such as initial cost, is varied (References 
30 and 31) • The differences in the attribute states of this one attribute 
varied in order to obtain indifference to each of the alterative systems will 
provide a tangible measure of the strength of preference between the 
alternative systems. 


6. Group Decision Models 

Throughout this section, decision-makers" has been consistently 
discussed in the plural. It is true that in American society, corporate and 
government (executive branch) decisions are ultimately the responsibility of 
one person, though the same cannot be said for either the legislative branch 
of government or the voting public. Thus, depending upon the context, it may 
be more appropriate to speak of decision-maker in the singular. Nevertheless, 
when one person holds the ultimate responsibility for the decision, this person 
may elect to delegate the decision-making responsibility to a group, or at 
least consider the preferences of several others prior to making the decision. 

Unfortunately, there presently exist no analytical models for group 
decision making that do not violate some intuitively desirable conditions. 

Arrow (Reference 30) was the first to demonstrate this fact. Extensive 
discussions of group decision making can be found in Fishburn (Reference 31), 
Luce and Raiffa (see Reference 25), and Sen (Reference 32). The best that can 
be done is to look at a range of group decision models, and where consensus of 
the/models is found, define that as the consensus of the group (References 33 
and 34) . 

The three group decision rules that will be considered in this report 
are the Rank Sum Rule, the Nash Bargaining Rule, and the Additive Utility 
Rule. (The Majority Decision Rule, which originally was considered for use in 
this analysis, was not employed because of unsolved theoretical problems that 
arise when more than two alternatives are involved). 

The Rank Sum Rule (References 30 and 35) in the slightly modified form 
proposed here, requires the calculation of the sum of the ordinal ranks for 
each alternative, with the alternative receiving the lowest rank sum being 
most preferred. Young (Reference 36) has stated four axioms that are neces- 
sary and sufficient for any collective choice rule to be equivalent to the 
Borda Rule. 

The Nash Bargaining Rule calculates the product of the utilities 
assigned by all the individuals to an alternative. The alternatives with 
greater utility product are more preferred, and from this a group preference 
order can be established. The Nash Bargaining Rule satisfies Nash's four 
axioms of "fairness" (Reference 37). As the number of decision-makers 
increase, the Nash utilities decrease because the individual utilities equal 
1.0. Hence, for even ten decision-makers, the Nash utilities are small. 

Without loss of generality, the Nash utilities can be re-scaled by taking 
the nth root of the product of the individual utilities, where n is the 
number of decision-makers in the group. 
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The modern formulation of the Additive Utility Rule is that of Harsanyi 
(Reference 38), The Additive Utility Rule averages the utility values 
assigned by the individuals to each alternative, with higher average utility 
values being more preferred* 

It should be re-emphasized that there is no theoretically compelling 
reason to use the results of any of these group decision rules, but they do 
provide information concerning the collective preferences of the decision- 
makers. 


B. RISK ANALYSIS 

1. Introduction 

Another element of the sensitivity analysis effort is that of risk 
analysis. Risk is defined as the possibility of loss or injury. This sub- 
section explains and illustrates the elements of risk analysis and describes 
how risk analysis is incorporated into the multiattribute decision model and 
into the sensitivity analysis. 


2. Risk-Analysis Elements 

Often the concept of risk analysis is introduced in the context of 
comparing two alternatives that have equal expected dollar value. An example 
is the following pair of alternatives: 

Option A: $1000 for sure. 

Option B: A 50-50 chance of zero dollars or of $2000. 

Although both options A and B have equal expected dollar values of $1000, 
they may not have equal expected utilities for some individuals. An indi- 
vidual's preferences between options A and B reveal his attitude toward risk 
in the range $0 to $2000: 

(1) An individual preferring A to B is characterized as risk-averse. 

(2) An individual preferring B to A is characterized as risk-prone. 

(3) An individual indifferent between A and B is characterized 

as risk-neutral. 

In the context of spaceborne power concepts, risk is apparent in the 
following hypothetical situation: 

Option C: Radiator area of 68 m^ with a technical development cost 

of $114 million. 

Option D: 50-50 chance of 108 or of 27 m^ of radiator area with a 

technical development cost of $114 million. 
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Although both options C and D have equal expected radiator area and equal 
development costs, individuals may exhibit different preferences, as with the 
previous dollar example. An individual preferring Option C to Option D is 
characterized as risk-averse, etc. 

Risk attitude implies a certain shape of the individual's utility func- 
tion and vice versa (see References 1 and 3). A risk-averse attitude for an 
attribute is equivalent to a concave utility function for that attribute. 
Also, risk-proneness is equivalent to a convex utility function; and finally, 
risk-neutrality is equivalent to a linear utility function. All three of 
these shapes are illustrated in Figure 2-4 for an increasing utility 
function. An increasing utility function exists for an attribute for which 
the decision-maker prefers higher values to lower values. 

The attitude of an individual toward risk varies with the range of 
outcomes. For example, few of us who would prefer Option B above would give 
$1, 000, 000 for sure for a 50-50 chance at zero or $2, 000, 000. Nevertheless, 
variation in individual attitude toward risk is evidenced by many motorists 
who drive from Los Angeles to Las Vegas to gamble (risk-prone), yet carry 
insurance on their automobiles (risk-averse). 


3. Incorporation of Risk in Multiattribute Decision-Making 

Risk has usually been incorporated in multiattribute decision 
making by taking the individual decision-maker's utility functions and 
probabilities of various outcomes and combining them to obtain an expected 
multiattribute utility for each decision alternative. Alternatives can then 
be ranked in order of expected multiattribute utility with the higher expected 
utility being the more preferred. The incorporation of risk in such a ranking 
occurs because the individual's attitude toward risk is embodied in the utility 
functions used to calculate expected utility. If he is risk-averse, then his 
multiattribute utility function will yield lower utility values for riskier 
alternatives. Similarly, if he is risk-prone, riskier alternatives will have 
higher utility values. 


C. CONCORDANCE 

It is important to determine the extent of agreement among interviewees 
as to the ranking of the alternative systems. To this end a statistic known 
as Kendall's Coefficient of Concordance was employed. This statistic varies 
between zero and one, with one corresponding to exact agreement among the 
judges and lower values indicating a greater degree of disagreement. The 
statistic has a known probability distribution. Thus, tests of significance 
can be performed. 

In the current analysis, the hypothesis that the set of rankings pro- 
duced by a number of judges are independent was tested. The null hypothesis, 
if accepted, would imply disagreement among judges. The more decisively one 
rejects this null hypothesis, the greater is the agreement, or concordance, 
among the judges. 
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Figure 2-4. Examples of Increasing Utility Functions for Different 
Risk Attitudes 


Kendall's Coefficient of Concordance, W, is given by the following 
equations : 


where 


W = 


YJ k 2 (N 3 - N) - k £ T i 

i=l 


A - 9 

s * 2 (R - ~ R > 

j=i J 


-is*! 

. = E /t 3 . - t.. / 1 2 

1 j=l \ 


k(N + 1 ) / 2 


and N * Number of alternatives. 


k * Number of judges. 

Rj s The sum of the ranks assigned to alternative j. 

t^j - Number of tied observations for rank j and judge i. 

The ranks, Rj, of tied observations are taken as equal to the average 
of the ranks they would have been assigned had no ties occurred. For example, 
suppose five alternatives, a through e, are ranked (from best to worst) d, a, 
c, e, b, with c and e tied. Ranks would be assigned as follows: d-1, a-2, 

c-3.5, e-3.5, b-5. 


2-15 


Table 2-1 gives the 5% and 1% significance points for S (the unnormal- 
ized statistic) and various values of k and N. When N > 7 one can use the 
fact that k(N - 1)W has, approximately, a chi-square distribution with N - 1 
degrees-of-freedom. When k(N - 1)W exceeds the critical significance point, 
the null hypothesis of independence of rankings, or lack of concordance among 
the judges is rejected. 


Table 2-1. Table of Critical Values of "S" in the Kendall 
Coefficient of Concordance 3 





N 



Additional values 
for N = 3 

k 

3 

4 

5 

6 

7 

k S 




Values at 

the 0.05 

Level of Significance 



3 



64.4 

130.9 

157.3 

9 

54.0 

4 


49.5 

88.4 

143.3 

217.0 

12 

71.9 

5 


62.6 

112.3 

182.4 

276.2 

14 

83.8 

6 


75.7 

136.1 

221.4 

335.2 

16 

95.8 

8 

48.1 

101.7 

183.7 

299.0 

453.1 

18 

107.7 

10 

60.0 

127.8 

231.2 

376.7 

571.0 



15 

89.8 

192.9 

349.8 

570.5 

864.9 



20 

119.7 

258.0 

468.5 

764.4 

1158.7 






Values at 

the 0.01 

Level of Significance 



3 



75.6 

122.8 

185.6 

19 


4 


61.4 

109.3 

176.2 

265.0 

■9 

103.5 

5 


80.5 

142.8 

229.4 

343.8 

14 

121.9 

6 


99.5 

176.1 

282.4 

422.6 

16 

140.2 

8 

66.8 

137.4 

242.7 

388.3 

579.9 

18 

158.6 

10 

85.1 

175.3 

309.1 

494.0 

737.0 



15 

131.0 

269.8 

475.2 

758.2 

1129.5 



20 

177.0 

364.2 

641.2 

1022.2 

1521.9 



a Source: Sidney Siegel, 

Nonparametric Statistics, McGraw-Hill, 

1956; 

P- 

286 (Reference 39). 
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SECTION III 


OBJECTIVES, CRITERIA, AND ATTRIBUTES 


A. INTRODUCTION 

In this section, the hierarchy of objectives, criteria, and attributes 
for evaluating and ranking alternative spaceborne power system concepts is 
presented. Desirable properties of attributes are described, followed by a 
statement of the original objectives to be used in evaluating alternative 
spaceborne power system concepts. Candidates for the objectives, criteria, 
and attributes are given. Some comments on steps toward a choice of the final 
attribute set and toward determination of scales for the selected attributed 
set conclude this section. 

There are several purposes to which this section is directed. The first 
is to explain the concept of a hierarchy of objectives, criteria, and attri- 
butes, and what properties are desired of this hierarchy. A second purpose is 
to provide background information in the form of the original SP-100 Project 
statement of objectives for the advanced concept alternatives. A final pur- 
pose is to detail the necessary steps to select the attribute set and its 
scales for use in the decision model. 


B. HIERARCHY OF OBJECTIVES, CRITERIA, AND ATTRIBUTES 

There is a structure that permits the transition from a broad statement 
of objectives to specific, measurable attributes that meet the needs of the 
decision model used to rank the alternatives (see Figure 2-3). Included in 
the hierarchy are an overall objective, subobjectives, criteria and attributes. 

Several properties are desired of this hierarchy. First, and most impor- 
tant, the hierarchy should lead to an appropriate ranking of alternatives, 
which is one that accurately reflects the preferences of the decision-maker. 
Second, the hierarchy should be reasonably easy to use. Ease of use is 
critical in order for the ranking to be achieved within time and cost 
limitations. Some aspects of ease of use include: 

(1) Ease of response for those required to provide preferences for the 
decision model. 

(2) Ease of obtaining performance data for alternatives with regard to 
the attributes. 

(3) Ease of carrying out the sensitivity analysis. 

The top level in the hierarchy is an overall statement of the objective 
for the power system concept alternatives (primarily in terras of basic require- 
ments). The overall objective for the project was to assess the potential of 
developing a nuclear powered source of energy for space applications. 
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The subobjectives provide distinct categories for the components of the 
overall objective* These components are chosen to facilitate further refine- 
ment of the hierarchy. Suggested categories for the subobjectives include 
economic, operational and technical objectives. 

The level below subobjectives contains criteria. The criteria must per- 
mit the quantification of performance of the alternatives with respect to the 
subobjectives. In other words, the criteria are the highest level elements in 
the hierarchy that are designed to be, or intended to be, quantifiable. For 
example, cost is a logical candidate for the criterion related to the economic 
subobjective . 

At the lowest level in the hierarchy are the attributes, which measure 
the extent to which each of the criteria are satisfied. To give an example, 
technical maturity may be an attribute to measure technical development 
requirements with respect to a risk criterion. 

The set of attributes to be employed when ranking advanced system 
alternatives must meet several technical requirements. It must be complete 
enough to include all of the factors that could significantly influence the 
decision, yet not so large as to overburden those who must provide prefer- 
ences. Attributes should be carefully selected to avoid redundancy or double 
counting of the system characteristics. The attributes selected should dif- 
ferentiate between systems by measuring only important advantages and disad- 
vantages inherent in the different types of technologies being considered. 

For instance, many of the cost factors may be represented by initial cost and 
life-cycle cost. Other attributes should measure major indicators such as 
technical, operational and organizational factors that impinge on the choice 
of advanced system alternatives. 


C. OBJECTIVES FOR ASSESSING SYSTEM CONCEPT ALTERNATIVES 

Four specific objectives of the MDA (Multiattribute Decision Analysis) 
are listed below. They are: 

(1) Determination of the spaceborne power system attribute values and 
relative weightings that reflect the preferences of decision makers 
in the public and private sectors relative to the nuclear industry 
(e.g. , safety, cost) . 

(2) Rank the system alternatives with respect to the overall objectives 
and attributes, based on the system and subsystem assessments. 

(3) Perform a sensitivity analysis on the rankings with regard to the 
system concept attribute values and the relative weightings. 

(4) Provide insights about possible combinations of nuclear tech- 
nologies toward construction of a proof-of-technology plan to 
carry out development of most promising technologies. 
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As a guideline for developing the attributes for the first objective, a 
list of requirements have been developed. Because many power system configura- 
tions and subsystem alternatives were being considered to overcome deficiencies 
of the baseline concepts, a comparison of system candidates on any meaningful 
basis requires equalizing as many of the external variables as possible. Thus 
the SP-100 Requirements were developed, which specify the system capabilities 
in terms of its size, power levels, mass, lifetime, and a number of other 
criteria. These requirements were used as design goals to synthesize, with 
the aid of models, the alternative system concepts evaluated. The final con- 
figurations are a result of the system requirements, subsystem characteristics, 
and control strategy trade-offs. The general SP-100 requirements are shown in 
Table 3-1. 

Perhaps the most critical parameters, in terms of the system design, 
were mass, temperature, and power level. Various parametric relationships 
between mass, power level, and temperature were used to define the various 
materials used and identify the feasible combinations of reactors, heat 
exchangers, and power conversion subsystems. Mass is obviously critical 
because of its sensitivity to a variety of design variables. Changes in 
temperature or materials can imply dramatic differences in mass. Because 
the power level was so interrelated with the other parameters, the assump- 
tion of a 100-kW level was made to provide a design baseline for the 
comparisons. By fixing the mass, and thus fixing a key dimension of the 
system, the synthesis of the systems was greatly simplified. On the other 
hand, issues such as growth capability were not included due to the lack of 
mission definition coupled with the assumption that a number of these 100-kW 
units could possibly be linked together to obtain higher power levels. Tem- 
perature was a key parameter since changes in hot-side temperatures define 
not only the mass, but the technology development. Increasing the operating 
temperatures for whatever benefits, in general, requires increasingly complex 
and longer-range technology development efforts to prove the concept. 

The design lifetime was also assumed to be 7 years in the analysis. 
However, in evaluating the choices among the alternative system concepts, 
the values associated with each alternative were in some cases related to the 
probable impact on lifetime. For example, in considering multiple start-up 
capabilities, some of the technical systems are more amenable to this capa- 
bility than other due to coolant freezing. In this sense, the alternative 
concepts were measured against their ability to meet the requirement. 

The safety requirements are a key concern and are stated in reference to 
a more detailed analysis of safety than presented here. Safety in this anal- 
ysis was defined as a multiple range of scenarios which at one extreme exceed 
the safety levels of current launch preparation, on-orbit operation, and at 
the other end are below these safety levels. 

A number of additional requirements were also considered but are not 
detailed here. These included load following capability, start-up, autonomy, 
reliability, survivability, dormancy, interfaces, reactor-induced and power- 
system-induced radiation, and size. 
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Table 3-1. Primary System Concept Requirements 


Requirements Value 

(1) System Mass 3000 kilograms 

(2) Design Lifetime 7 years 

(3) Safety Shall meet all defined requirements 

(4) Power Output 100 kilowatts 

Additional Requirements 

(5) Power Distribution 

(6) Load Following Capability 

(7) Start-up Characteristics 

(8) Autonomy of System 

(9) Reliability 

( 10) Survivability 

(11) Dormancy 

(12) Interfaces (Electrical, Command/ 

Data/Telecommunications) 

(13) React or- Induced Radiation to Payload 

(14) Power-System- Induced Thermal Radiation 
(13) Size 


The SP— 100 requirements list was used to begin to define the heirarchy 
of objectives, criteria, and attributes for ranking alternatives. The first 
task was to separate the objectives to be used in the ranking methodology for 
alternatives from those objectives that are fixed requirements or constraints. 

Good candidates for constraints included requirements (1), (2), (4), 

(15), and (12) through (14). They could be treated as constraints by 
requiring any system concept to meet them before being accepted for ranking 
with regard to the remaining objectives. Good candidates for attributes 
included requirements (3) and (6) through (11) because they can be used 
effectively to differentiate between alternative systems. 

The objectives of cost minimization, high technical maturity, safety, 
and performance were also candidates to aid in the definition of the 
hierarchy. Objectives, criteria, and attribute sets are discussed below. 


D. OBJECTIVES, CRITERIA, AND ATTRIBUTE SETS 

Several sets of candidates for use as objectives, criteria, and attri- 
butes were developed. While reviewing these sets, it was noted that there 
were two possibly conflicting objectives for the set chosen for use with the 
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decision model. The criteria and attribute set had to be complete enough to 
capture the reality of the problem, yet not so large that it overburdened 
those people who had to provide their preferences nor those who exercised 
the decision model and carried out the sensitivity analysis. 

The candidate sets of objectives, criteria, and attributes were reviewed 
by Project staff at JPL and representatives from Los Alamos National Labora- 
tories and NASA Lewis Research Center. After several iterations, a set for 
use in the ranking was chosen. 

The hierarchy chosen is shown in Figure 3-1. This set includes a single 
overall objective, eight subobjectives (safety, payload, survivability, opera- 
tional, technical, schedule, and economic), eight criteria, and eight attri- 
butes. With eight attributes, the ranking and sensitivity analysis proved 
manageable. Also, after the interviews, no significant attribute was found 
to be missing from the set chosen, based on the information available at that 
time. Estimated development cost and production cost were deemed to be desir- 
able, but insufficient information was available for estimating these elements 
and so they were not included in the formal analysis. 


E. DISCUSSION OF ATTRIBUTES 

Safety was characterized in terms of a scenario scale that ranged from 
0-10 where each point on the scale is described by a brief statement regard- 
ing that safety level. In the best case, the safety level would exceed that 
of present launch vehicles. The scale itself was divided into a number of 
subdimensions including pre-launch, launch, on-orbit operation, and re-entry. 

Survivability was characterized in terms of a scenario scale attribute 
called estimated likelihood of surviving threats at required levels. The pri- 
mary concern here was for man-made threats as opposed to those in the natural 
environment, such as meteorites. It was assumed that all the systems were 
comparable in terms of armor to protect against meteorites. 

The operational aspects of the system concepts were captured with three 
attributes: dormancy capability, radiator area, and likelihood of meeting the 

reliability requirements. Again, both dormancy capability and likelihood of 
meeting the reliability requirements were measured, using a scenario scale 
from 0 - 10. The radiator area was measured in terms of square meters. 

The technical elements of the comparison were characterized by another 
descriptive measure called producibility . Producibility measures the modu- 
larity, fabricability , and level of interfacing involved in the construction 
of the system. The producibility was measured on a 1 - 10 scale in a similar 
manner to technical maturity with points on the scale described with brief 
statements. 

The schedule elements of the evaluation were characterized using a 
descriptive measure called technical maturity. The technical maturity was 
characterized in terms of a 0 - 10 scale where points on the scale are 
represented by brief statements describing each level. 
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Three cost measures were identified: estimated cost to prove technical 

feasibility (at a pre-defined level), development cost, and production cost. 
Development cost was desired because it tends to scope the overall project 
cost. Production cost was of interest due to the economies of scale possible 
with the production of large numbers of power systems and the fact that large 
development costs could possibly be outweighed by low unit costs. As mentioned 
earlier, development cost and production cost were not believed to be signifi- 
cant in affecting the overall rankings and thus were not included. The key 
cost attribute used was the estimated cost to prove technical feasibility. 

This value was more appropriate because the overall scope of this effort was 
to provide input into the development of a plan to demonstrate technical 
feasibility. The costs were measured in 1983 dollars because all of the 
interviews were conducted in 1983. This cost attribute was considered most 
directly related to the ranking of a system concept. 


F. DETERMINATION OF ATTRIBUTE SCALES 

In order for the decision model to be applied in the ranking effort, a 
scale for each attribute used had to be developed. Each scale required a unit 
measure and upper and lower bounds. For example, the attribute estimated cost 
to prove technical feasibility, 1983 dollars was the unit of measure, and $114 
million and $240 million dollars were the lower and upper bounds. Because the 
nature of the task involved technology assessment and the synthesis of con- 
ceptual representations of these systems, only subsystem parametric data were 
available for the most part. As a result, the majority of attributes were 
characterized, using descriptive scenario scales to develop the ranges neces- 
sary to discriminate between systems. The list of attributes chosen with the 
ranges for cost and performance is given in Table 3-2. 

The upper and lower bounds for each attribute had to be determined so 
that all alternatives had performance levels that fit within these bounds. 

If a performance level had fallen outside one of these bounds, the utility 
of that performance level could not have been calculated. 
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Table 3-2. Attributes with Their Ranges 



Attribute 3 


Range 

(1) 

Safety 

Level 

3 to 8 (scenario) 

(2) 

Radiator Area* 5 

27 to 

108 m^ 

(3) 

Design Reliability 

Level 

2 to 10 (scenario) 

(4) 

Technical Maturity 

Level 

3.8 to 7.8 (scenario) 

(5) 

Estimated Cost to Reach 
Technical Feasibility 

$114 to 240 million (1983 dollars) 

(6) 

Survivability 

Level 

5 to 10 (scenario) 

(7) 

Dormancy Capability 0 

Level 

2 to 10 (scenario) 

(8) 

Producibility 

Level 

3 to 8 (scenario) 

a See Appendix A for the scale 
(4), and (6) through (8). 

definitions 

for attributes (1), (3), 


^Assumes larger radiators deployable. 

c Load following comparable for all systems via shunt. 


SECTION IV 


ALTERNATIVES AND STATE DATA 


A. INTRODUCTION 

This section briefly lists the sixteen alternative system concepts 
ranked by this study and gives the state data for each concept for the eight 
attributes. The attributes are described in Section III. 


B. ALTERNATIVE SYSTEM CONCEPTS 

The systems included seven heat-pipe cooled and seven liquid-metal 
cooled systems with a variety of dynamic and static power conversion systems. 
One gas-cooled system and an in-core system were also examined. The conver- 
sion systems included Brayton, Stirling, Rankine, thermoelectric, thermophoto- 
voltaic, thermionic, and AMTEC technologies. The sixteen system concepts are 
listed in Table 4-1 along with their acronyms used to identify the systems in 
the interview process questionnaire (Appendix A) and in the tables of ranking 
results (Section VI), respectively. Performance requirements for all sixteen 
systems are given in Table 3-1. 



Table 4-1. 

Alternative System Concepts with 

Abbreviations 


System Concept 

System Concept 
Abbreviation 

1 . 

Liquid-metal cooled/out-of-core thermionic 

LOCTP 

2. 

Liquid-metal cooled/Bray ton 

LBO 

3. 

Liquid-metal cooled/Stirling 

LSH 

4. 

Liquid-metal cooled/Rankine 

LRL 

5. 

Liquid-metal cooled/AMTEC 

LAP 

6. 

Liquid-metal cooled/Thermoelec trie 

LTEP 

7. 

Gas-cooled/Bray ton 

GBH 

8. 

Heat-pipe 

cooled/out-of-core thermionic 

HOCTP 

9. 

Heat-pipe 

cooled/Brayton 

HBO 

10. 

Heat-pipe 

cooled/Stirling 

HSH 

11. 

Heat-pipe 

cooled/Rankine 

HRL 

12. 

Heat-pipe 

cooled/AMTEC 

HAP 

13. 

Heat-pipe 

cooled/ thermophotovoltaic 

HTPVP 

14. 

Heat-pipe 

cooled/ thermoelectric ( 1380K) 

HTEP 

15. 

Heat-pipe 

cooled/ thermoelectric ( 1250K) 

HTEPa 

16. 

In-core- thermionic 

ICT 
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C. SYSTEM CONCEPT ATTRIBUTE STATE DATA 

The attribute state data for the sixteen concepts were developed in June 
and July 1983. The data are presented in Table 4-2. The details of the sub- 
jective scales for safety, technical maturity, design reliability, dormancy, 
survivability, and producibility are given in Section III. To illustrate 
those concepts that perform well irrespective of the relative importance of 
the attributes, consider Table 4-3. The attributes within 10% (of the range) 
of the best state of each attribute are marked. If all the attributes were 
equally important, the systems with the most checkmarks would be the preferred 
concepts. Table 4-3 shows, independent of the value model, that the heat-pipe 
thermoelectrics (HTEP, HTEPa) and Stirling concepts (HSH, LSH) rate highly on 
a number of attributes. This table is helpful in explaining the results of 
the ranking procedure. 

These data were the culmination of effort by the SP-100 Technology 
Assessment Working Group of the SP-100 Project and reflect a detailed analysis 
of each of the major subsystems and their components. As mentioned earlier, 
much of the data collected were of a parametric form that were used with 
models and the requirements to synthesize the sixteen systems presented here. 
It should be noted that these values reflect a great deal of technical judg- 
ment because the majority of scales were subjective. However, the relative 
values among the system concepts are believed to be valuable information. 

The major difficulties occurred with the assessments of cost and technical 
maturity. There were a large number of uncertainties in the cost estimates 
because the totals were dominated by the reactor development costs. The 
technical maturity of each system was determined by assigning weights to each 
of the major components within each subsystem and then each subsystem. Each 
component and subsystem was then assigned a technical maturity value from the 
scale in Appendix A and a linear weighting was performed to calculate an 
overall technical maturity value assigned to the system as a whole. 
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Alternative 
System Concept a 




Attribute 




Safety 

Radiator 

Area 

Design 

Reliab. 

Technical 

Maturity 

Est. Cost/ 
Tech. Feas. 

$M 

Survivability 

Dormancy 

Producibility 

LOCTP 

7 

42 

8 

6.0 

193 

7 

4 

6 

LBO 

7 

100 

6 

7.0 

198 

6 

4 

4 

LSH 

7 

31 

7 

7.8 

124 

7 

4 

5 

LRL 

7 

27 

4 

6.9 

140 

5 

2 

3 

LAP 

7 

60 

4 

6.9 

114 

6 

2 

5 

LTEP 

7 

80 

9 

7.2 

143 

8 

5 

8 

GBH 

3 

50 

2 

3.8 

213 

5 

9 

4 

HOCTP 

8 

42 

8 

6.0 

200 

8 

8 

6 

HBO 

8 

107 

7 

7.0 

190 

7 

8 

4 

HSH 

8 

31 

8 

7.8 

124 

8 

8 

5 

HRL 

8 

27 

5 

6.9 

160 

6 

4 

3 

HAP 

8 

60 

5 

6.7 

114 

7 

4 

5 

HTPVP 

8 

108 

5 

3.9 

240 

5 

9 

7 

HTEP 

8 

67 

10 

6.3 

135 

10 

10 

8 

HTEPa 

8 

80 

10 

7.4 

135 

10 

10 

8 

ICT 

6 

38 

7 

7.6 

170 

9 

10 

7 


a LOCTP = Liquid-metal cooledlout-of-core thermionic 
LBO = Liquid-metal cooledfBrayton 

LSH = Liquid-metal cooledfStirling 

LRL = Liquid-metal cooledIRankine 

LAP = Liquid-metal cooledjAMTEC 

LTEP = Liquid-metal cooledlthermoelectric 
GBH = Gas-cooledIBrayton 

HOCTP ■ Heat-pipe cooled/out-of-core thermionic 

HBO = Heat-pipe cooledIBrayton 

HSH = Heat-pipe cooled/Stirling 

HRL = Heat-pipe cooled/Rankine 

HAP = Heat-pipe cooledfAMTEC 

HTPVP = Heat-pipe cooledlthermophotovoltaic 
HTEP = Heat-pipe cooledlthermoelectric (1380K) 
HTEPa = Heat-pipe cooledlthermoelectric (1250K) 
ICT = In-core thermionic 


Table 4-2. System Database for Sixteen System Concepts 


4-3 






SECTION V 


INTERVIEWS 


A. INTRODUCTION 

The methodology described in Section II requires preference information 
from individuals as well as attribute state data to produce a ranking of sys- 
tems. The preference information required for each individual interviewed 
includes a scaling constant and a utility function for each attribute. Inter- 
viewees were sought who had significant knowledge of, and interest in, space- 
borne nuclear power system concepts and who were regarded as decision makers 
within their organizations. 

This section lists the organizations interviewed to obtain preference 
data and gives examples of the questions posed to them. (The full set of ques- 
tions is contained in Appendix A) A summary of the interview results is also 
given in this section. 


B. INTERVIEWEES 

The desired interviewees were persons who would either have a direct 
role in the ultimate development of the concepts or who acted as advisors in 
the decision-making process. Representatives were sought from a variety of 
organizations with: 

(1) Ongoing research and development programs in advanced power 
conversion systems. 

(2) A proven record of achievement in the research and development of 
nuclear power systems. 

(3) An understanding of space environment issues that have direct 
impact on developing nuclear power technologies for space 
applications. 

These individuals represented four distinct areas: 

(1) Safety. This group was concerned with a range of safety issues 
from ground development through launch, on-orbit operation, and 
re-entry. 

(2) Systems Definition and Design. This group was concerned with the 
design issues and options involved in the development and deploy- 
ment of the technology. 

(3) Technology Assessment. This group was involved in assessing the 
technical issues facing the demonstration of technical feasibility 
for such power systems. 
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(4) Mission Analysis. This area involved the concerns of possible 
mission users who would utilize the system concepts. 

Altogether, 11 people were interviewed between July 7, 1983, and July 
22, 1983. The organizations represented included the Air Force Weapons 
Laboratory, Jet Propulsion Laboratory, Los Alamos National Laboratories, and 
NASA-Lewis Research Center. They included four individuals from the safety 
area, three from the systems definition and design category, three from the 
technology assessment working group, and one from the mission analysis cate- 
gory. Accordingly, 11 complete interviews form the corpus of the analysis. 

The representation of members in the sample was constituted from an ini- 
tial survey of representatives derived from conference agendas, personal con- 
tacts, and referrals. This "snowball" sampling approach was further refined 
during the interviews as additional recommendations were made. These recommen- 
dations were then reviewed for inclusion in the study. While this sample is 
not a random one, there were numerous individuals who simply had to be inclu- 
ded because they had played a key role in some aspect of the advanced research. 
Using a random sampling design and possibly omitting them from the survey would 
have left serious gaps in the results of the study. Furthermore, a larger, 
random sample would tend to move the results toward some "average" set of 
responses. The aim of this study was to survey those at the leading edge of 
the advanced concepts development to obtain an informed, critical response as 
opposed to an average or typical response. Although more interviews might 
have been desirable, the time and resources to accomplish them were not 
available. 


C. INTERVIEW PROCESS 

The selected personnel were asked to provide their inputs to the rank- 
ings during one-hour interviews although, in fact, the interviews ranged from 
60 to 100 min with an average of 75 min and a median of 75 min. These sessions 
were structured to acquire the interviewee's utility functions and scaling con- 
stants with regard to the attributes chosen for the purpose of ranking alterna- 
tive advanced vehicle systems. 

There were five steps in the decision-analysis interview, as shown in 
Figure 5-1. The first step provided an introduction to the interview and 
afforded the opportunity to have the interviewee's questions about the pro- 
cess answered. Next, the interviewee's utility function for each attribute 
was obtained by asking a series of preference questions. Following that, 
independence was checked by asking if the responses to those questions would 
vary with changes in the levels of the other attributes (i.e., attributes 
other than the one whose utility function was being assessed). The fourth 
step in the interview involved having the interviewee rank the attributes in 
order of importance. This provided a consistency check to aid with the final 
step, the acquisition of the interviewee's scaling constant for each attri- 
bute. The ranking of attributes helped guide the responses to the questions 
on scaling constants. 
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D. SAMPLE QUESTIONS 

Sample questions for the interviews are illustrated by Figures 5-2, 

5-3, and 3-4, Figure 3-2 contains a sample question used to obtain informa- 
tion that enabled the construction of the individual's utility function for 
the attribute "radiator area." Figure 5-3 contains a sample question for the 
ranking of attributes in order of importance, while Figure 5-4 shows a sample 
question for obtaining the scaling constant for an attribute. The full ques- 
tionnaire used is contained in Appendix A in of this report. 


E. INTERVIEW PROCESS REFERENCES 

The use of interviews in the decision analysis process is well estab- 
lished and documented. Excellent descriptions of decision analysis with 
interviews are provided by Raiffa, Schlaifer, and Winkler (see References 3, 

5, and 24). References on decision-analysis interviews particulary well-suited 
to the manager include Brown, Kahr, and Peterson (see Reference 21) and Huber 
(Reference 42). Chapter 4 of Huber's recent book (Reference 42) contains two 
case studies involving multiattribute decision-tqaking. The authoritative book 
by Keeney and Raiffa (see Reference 1) contains a variety of case studies in 
multiattribute decision-making. Most of these cases are in Chapter 7, but 
one, involving airport development, described in detail in Chapter 8, includes 
responses to interview questions on utilities, independence, and scaling con- 
stants. Additional cases can be found in Feinberg, et al. (References 40 and 
41). 


F. INTERVIEW RESULTS 

On the whole, the interviews went rather smoothly. All interviewees 
were able to provide the information needed to form their attribute utility 
functions and scaling constants. The average length of the interview was 75 
min with the longest session completed in 100 min and the shortest in 60 min. 
There were five interviews (46%) that took 70 min or less; four interviews 
(36%) between 70 and 85 min; and two interviews (18%) that took 100 min. All 
11 interviews were completed within 100 min, with nine (82%) less than 90 min. 

The responses for the interviewees to the questions designed to elicit 
information needed to determine their attribute utility functions are summar- 
ized in Table 5-1 for the entire sample. Table 5-2 (a through c) shows the 
results by group (the mission analysis results are not shown because one 
person represented that area). As shown, there was a willingness in many 
cases to take a risk to obtain good (rather than average) technical maturity 
and safety levels. The safety group tended to be risk-averse to large radia- 
tor areas, poor technical maturity, and survivability. The systems area was 
risk-averse to low technical maturity and low survivability. The technology 
assessment area was generally neutral about cost, safety, and radiator area 
with different risk attitudes for survivability, dormancy, and producibility . 
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ATTRIBUTE: RADIATOR AREA 



• FOR WHICH VALUE OF THE "SURE THING" ARE YOU 
INDIFFERENT BETWEEN THE "SURE THING" AND THE 
"GAMBLE"? 

INDIFFERENCE POINT 

• IF YOU KNEW THAT ALL OTHER ATTRIBUTES WERE AT 
THEIR WORST STATES? 

INDIFFERENCE POINT 

• IF YOU KNEW THAT ALL OTHER ATTRIBUTES WERE AT 
THEIR BEST STATES? 

INDIFFERENCE POINT 


Figure 5-2. Sample Interview Question, Radiator Area 


5-5 



Figure 5 - 3 . Sample of Interview Question, Order of Attribute Importance 




































Figure 5-4. Sample Interview Question, Importance of Radiator Area 
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Table 5-2. Preference Data for Interviews 

(b) System Definition and Design Area 
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Responses to the questions asking interviewees to rank the importance of 
each of the attributes are summarized for each group and the entire sample in 
Table 5-3. The ranking for each group was determined by taking the sum of the 
individual rankings within that group and placing the lowest sum as first in 
rank, the next lowest sum second, and so on. 

Overall, initial safety and technical maturity were most important, and 
radiator area and cost least important (see also Table 5-4). It is interesting 
to note that some individuals ranked safety much lower than other attributes. 
This was due (primarily) to the perception that safety is a secondary issue 
(or non-issue) until it can be shown that the system is technically feasible. 
The mission analysis area (representing users to some extent) was less inter- 
ested in cost and technical maturity than the more operational attributes like 
reliability, survivability, and producibility . 


Table 5-3. Preference Data from Interviews, Importance of Attributes 




Rank Sum Rule 

Rankings 


Attribute 

Safety 

Area 

Systems Definition 
and Design Area 

Technology 

Assessment 

Mission 

Analysis 

Safety 

1 

i 

2-3 

i 

Radiator Area 

7-8 

8 

6-8 

4-6 

Design Reliability 

3 

3 

2-3 

2-3 

Technical Maturity 
Estimated Cost to 
Reach Technical 

2 

2 

1 

7-8 

Feasibility 

7-8 

6-7 

6-8 

7-8 

Survivability 

5 

4-5 

5 

4-6 

Dormancy 

6 

6-7 

6-8 

4-6 

Producibility 

4 

4-5 

4 

2-3 
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Table 5-4. Ranking of Attribute Importance 



Number of Times Rated 


Attribute 

Most Important Least 

Important 

Safety 

7 

i 

Radiator Area 

0 

4 

Design Reliability 

0 

0 

Technical Maturity 

4 

0 

Estimated Cost to Prove 
Technical Feasibility 

0 

5 

Survivability 

0 

0 

Dormancy 

0 

2 

Producibility 

0 

1 
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SECTION VI 


RANKING ANALYSIS AND DISCUSSION 


A. OVERVIEW OF THE DECISION ANALYSIS RESULTS 


The results of 11 successfully conducted interviews were analyzed by 
several different methods. Preference data were elicited from the interviewees 
on eight attributes for use in a multiattribute decision-analysis model. 

The 11 interviews were classified into four areas, with three to four 
interviews in a group. The mission analysis area was represented by one 
individual. The four areas were generically classified as: 


Group 1: 
Group 2: 
Group 3: 
Group 4: 


Safety 

Systems Definition and Design 
Technology Assessment 
Mission Analysis 


The rankings were developed by interviewee and by group. Three group 
decision rules were used for the groups: (1) The Additive Rule, (2) The Nash 

Bargaining Rule, and (3) The Rank Sum Rule. 


B. MATEUS COMPUTER RUNS 

A total of 40 MATEUS (Mult iATtribute Evaluation of UtilitiejO runs were 
made to calculate preferences from the data of the multiattribute decision 
analysis interviews. The MATEUS Computer Program is given in Appendix B. The 
runs calculated both individual and group preferences. A single run calcula- 
ted the preferences for a single group and for each of the interviewees of 
that group. The 40 MATEUS runs were composed of four runs of the nominal data 
and twelve variations (three runs each) on the nominal data. Comparable runs 
for the fourth group were not made because only one individual represented 
that group. However, a fourth run was included using the baseline nominal 
data (see Set 1 below). 

The 40 runs, in sets of three for the three groups, are identified as 
follows : 

(1) Set 1 is the set of runs with the nominal data and are referred to 
as the N0M/5/MULT Set. The attribute scaling constants are deter- 
mined with the other attributes set at nominal states, 5 points 
are used for the piece-wise linear fit to the attribute utility 
functions, and the multiplicative form of the Keeney-Raif f a 
methodology is used. 
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(2) Set 2 is identified as the N0M/5/LIN Set. Set 2 is identical to 
the N0M/5/MULT Set (Set 1), except that the attribute scaling 
constants are normalized so that their sum is 1.0, and the Linear 
Form of the Keeney-Raif fa methodology is used. 

(3) Set 3 is identified as the NOM/3/MULT Set. Set 3 is identical to 
the N0M/5/MULT Set (Set 1), except that a 3-point fit rather than 
a 5-point fit is used for the piece-wise linear fit to the 
attribute utility functions. 

(4) Set 4 is identified as the WORST/3/MULT Set. Set 4 is identical 
to the NOM/3/MULT Set (Set 3), except that the attribute utility 
functions were elicited with the other attributes set at their 
worst states. 

(5) Set 5 is identified as the BEST/3/MULT Set. Set 5 is identical to 
the N0M/3/MULT Set (Set 3), except that the attribute utility 
functions were elicited with the other attributes set at their 
best states. 

(6) Set 6 is identified as the SAFETY Set. Set 6 is identical to 
the N0M/5/MULT Set (Set 1), except that the attribute state for 
safety (Attribute #1) is fixed at 3 for all systems. The effect 
of fixing an attribute at its worst state is to remove it, and its 
contribution, from the analysis. This reveals the sensitivity of 
the rankings to the attribute. 

(7) Set 7 is identified as the RADAREA Set. Set 7 is identical to the 
NOM/ 5 /MULT Set (Set 1), except that the attribute state for 
radiator area (Attribute #2) is fixed at 108m^ for all systems. 

(8) Set 8 is identified as the DESREL Set. Set 8 is identical to the 
NOM/ 5 /MULT Set (Set 1), except that the attribute state for design 
reliability (Attribute #3) is fixed at 2 for all systems. 

(9) Set 9 is identified as the TECHMAT Set. Set 9 is identical to 
the N0M/5/MULT Set (Set 1), except that the attribute state for 
technical maturity (Attribute #4) is fixed at 3.8 for all systems. 

(10) Set 10 is identified as the FEASCOST Set. Set 10 is identical to 
the NOM/ 5 /MULT Set (Set 1), except that the attribute state for 
estimated cost to prove technical feasibility (Attribute #5) is 
fixed at $240 million for all systems. 

(11) Set 11 is identified as the SURV Set. Set 11 is identical to the 
N0M/5/MULT Set (Set 1), except that the attribute state for 
survivability (Attribute #6) is at 5 for all systems. 

(12) Set 12 is identified as the DORMANCY Set. Set 12 is identical to 
the NOM/5/MULT Set (Set 1), except that the attribute state for 
dormancy (Attribute 7) is fixed at 2 for all systems. 
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(13) Set 13 is identified as the PRODUC Set* Set 13 is identical to 
the N0M/5/MULT Set (Set 1), except that the attribute state for 
producibility (Attribute 8) is fixed at 3 for all systems* 


C. MULTIATTRIBUTE RESULTS FOR NOMINAL ATTRIBUTE DATA 

Preference data was elicited from the interviewees on eight attributes 
for use in a multiattribute decision analysis, with twelve variations on the 
nominal data to examine the robustness of the multiattribute states on the 
rankings. The methodology for the multiattribute decision analysis model is 
the Keeney-Raif fa Methodology, which is discussed in Section II. Section VI 
discusses the multiattribute results specifically for the nominal data. The 
nominal data is defined to be the data gathered in the interviews with the 
5-point piece-wise linear fit to the interviewee utility functions, the scal- 
ing constants determined with the other attributes at nominal states, and the 
alternative system data unmodified (Set 1: NOM/ 5/MULT) . 

Table 6-1 gives the rankings for the system concepts for all 11 inter- 
viewees for Set 1 (N0M/5/MULT) . Table 6-1 shows that the heat-pipe thermo- 
electric reactor systems (HTEP, HTEPa) were preferred followed by the heat- 
pipe Stirling (HSH) system and then a split over the in-core thermionic (ICT) 
versus the heat-pipe out-of-core thermophotovoltaic (HOCTP) for fourth place. 
The liquid-metal thermoelectric (LTEP) and Stirling (LSH) are next followed by 
the heat-pipe AMTEC (HAP). The least preferred systems are the liquid-metal 
Brayton (LBO) , liquid-metal Rankine (LRL), and gas-cooled Brayton (GBH) system. 

When the individual rankings are aggregated into group rankings, the 
rank order becomes somewhat more apparent. See Table 6-2, which was con- 
structed from the results of the three group decision rules. The heat-pipe 
therraoelectrics (HTEP, HTEPa) still rank first followed by the heat-pipe 
Stirling (HSH), the in-core thermionic (ICT) and heat-pipe out-of-core 
thermionic (HOCTP). The least preferred systems are still the liquid-metal 
Brayton (LBO), liquid-metal Rankine (LRL), and gas-cooled Brayton (GBH). 

Table 6-3 gives the rank ordering for the alternative systems accord- 
ing to each of the eight attributes. Each of the attributes was varied in 
developing the designs that determined the attribute states of the sixteen 
alternative systems. It is not apparent from Table 6-3 that any one of the 
attributes can account for the multiattribute results with the nominal data, 
design reliability, survivability, dormancy, and producibility appear to have 
the greatest effect in the contributions to the upper-level rankings. 


D. RESULTS OF VARIATIONS ON MULTIATTRIBUTE NOMINAL DATA 

Runs with thirteen variations on the multiattribute nominal data (Sets 2 
through 13) were made to examine the variations of the results to the multi- 
plicative model form, to the preferences of the interviewees, and to the 
specified states of the alternative systems. Only the group decision results 
will be discussed because the variations are all of second order, and the 
group decision rules best summarize the variations in the rankings. The 
comparisons are summarized in Tables 6-4a, b, and c. 
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Table 6-1. Rankings for All Individuals for Set 1 (N0M/5/MULT) 




Area 

l a 



Area 2 



Area 3 


Area 4 

Interview No. 

System 

Concept 13 

5 

10 

2 

i 

8 

6 

9 

4 

7 

3 

ii 

LOCTP 

10 

11 

10 

9 

12 

10 

13 

11 

8 

12 

12 

LBO 

14 

13 

14 

14 

14 

13 

14 

14 

14 

14 

15 

LSH 

7 

8 

8 

5 

8 

5 

7 

8 

6 

6 

11 

LRL 

13 

15 

15 

13 

15 

14 

15 

13 

13 

15 

14 

LAP 

12 

14 

13 

n 

13 

12 

12 

12 

12 

13 

13 

LTEP 

5 

4 

5 

6 

10 

6 

4 

6 

4 

8 

9 

GBH 

16 

16 

16 

16 

16 

16 

16 

16 

16 

16 

16 

HOCTP 

4 

5 

6 

7 

4 

7 

6 

■ 

B 

5 

4 

HBO 

9 

9 

7 

10 

6 

8 

9 

9 

10 

9 

5 

HSH 

2 

2 

3 

3 

3 

2 

3 

3 

5 

3 

3 

HRL 

11 

11 

11 

12 

9 

11 

10 

7 

11 

11 

6 

HAP 

8 

8 

9 

8 

7 

9 

5 

5 

9 

7 

7 

HTPVP 

15 

15 

12 

15 

D 

15 

11 

15 

15 

10 

8 

HTEP 

3 

3 

2 

2 

2 

3 

2 

2 

2 

2 

2 

HTEPa 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

ICT 

6 

6 

E 

o 

5 

4 

8 


3 

4 

10 

a See page 6-1 for 
bSee Table 4-1 on 

group names, 
page 4-2 for 

system 

names. 
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Table 6-3. Rankings by Each of Eight Attributes 


System 

Safety 

Rad. 

Area 

Design 

Reliab. 

Tech. 

Maturity 

Feas. 

Cost 

Surviv. 

Dorman. 

Prod. 

LOCTP 

9-14 

6-7 

4-6 

13-14 

12 

7-10 

10-14 

6-7 

LBO 

9-14 

14 

10 

6-7 

13 

11-13 

10-14 

12-14 

LSH 

9-14 

3-4 

7-9 

1-2 

3-4 

7-10 

10-14 

8-11 

LRL 

9-14 

1-2 

14-15 

8-10 

7 

14-16 

15-16 

15-16 

LAP 

9-14 

9-10 

14-15 

8-10 

1-2 

11-13 

15-16 

8-11 

LTEP 

9-14 

12-13 

3 

5 

8 

4-6 

9 

1-3 

GBH 

16 

8 

16 

16 

15 

14-16 

4-5 

12-14 

HOCTP 

1-8 

6-7 

4-6 

13-14 

14 

4-6 

6-8 

6-7 

HBO 

1-8 

15 

7-9 

6-7 

11 

7-10 

6-8 

12-14 

HSH 

1-8 

3-4 

4-6 

1-2 

3-4 

4-6 

6-8 

8-11 

HRL 

1-8 

1-2 

11-13 

8-10 

9 

11-13 

10-14 

15-16 

HAP 

1-8 

9-10 

11-13 

11 

1-2 

7-10 

10-14 

8-11 

HTPVP 

1-8 

16 

11-13 

15 

16 

14-16 

4-5 

4-5 

HTEP 

1-8 

11 

1-2 

12 

5-6 

1-2 

1-3 

1-3 

HTEPa 

1-8 

12-13 

1-2 

4 

5-6 

1-2 

1-3 

1-3 

ICT 

15 

5 

7-9 

3 


3 

1-3 

4-5 
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Table 6-4. Multiattribute Decision Analysis: Summary of System Concept Ranking 

(a) Results for Safety Group 
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(12) N0M/5/MULT/ 

DORM @ 2 9-10 13 5-6 13-14 12 4 16 5-6 9-10 


Table 6-4. Multiattribute Decision Analysis: Summary of System Concept Rankings 

(b) Results for Systems Definition and Design Group 
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(13) NOM /5/MULT/ 

PROD @ 3 11 13-14 4-7 14-15 12 9-10 16 4-7 6-8 1 9-10 5 13-15 3 1-2 6-8 



Table 6-4. Multiattribute Decision Analysis: Summary of System Concept Rankings 

(c) Results for Technology Assessment Group 
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(13) N0M/5/MULT/ 

PK°D @ 3 11 14-15 4-5 13 12 6-9 16 7-8 8-9 2 9-10 4-7 14-15 


The data for Set 2 (N0M/5/LIN) were derived from the data of Set 1 
( NOM/ 5/MULT) by normalizing the sum of the attribute scaling constants to 
1.0 for all interviewees. This reduces the Keeney Multiplicative Form to 
the Linear Form, where the attributed scaling constants are simple weighting 
factors, and the multiattribute model is just the weighted sum of the attri- 
bute utilities. Linearizing the model makes a significant difference in the 
form of the model because the sum of the attribute scaling constants for the 
nominal data for the 11 interviewees range from a low of 2.70 (master scaling 
constant = -0.965791) to a high of 4.80 (master scaling constant = -0.999696) 
with a mean value of 3.516. Even with this significant change in the scaling 
constants, the results prove to be very robust, with very little change in 
ranking as shown in Table 6-4. 

The data for Set 3 (N0M/3/MULT) were derived from the data of Set 1 
(N0M/5/MULT) by using only the end points and the midpoint of the attribute 
utility functions. The results of using Set 3 serve two purposes: (1) to 

examine the sensitivity of the results to the coarseness of the piece-wise 
linear approximation to what is almost certainly a smooth function, and (2) to 
use as a reference for examining the results from using the data of Set 4 and 
Set 5. The results of using the data for Set 3 are shown in Table 6-4. The 
results of using the data for Set 3 are virtually identical to the results of 
using the data for Set 1, with only small changes in ranking for a few appli- 
cations of the group decision rules. 

The data for Set 4 (WORST/ 3 /MULT) were derived from the interview data, 
using the attribute utility functions obtained when the interviewees were asked 
to assume that all other attributes were at their worst states. Placing the 
other attributes at their worst states made some of the alternative systems so 
undesirable that some interviewees were unable to respond with answers that 
could be translated into attribute utility functions. Where runs could be made 
to determine rankings with the group decision rules, once again only minor 
changes in ranking for a few applications of the group decision rules occurred 
(Table 6-4). 

The data for Set 5 (BEST/3/MULT) were derived from the interview data, 
using the attribute scaling constants obtained when the interviewees were asked 
to assume that all other attributes were at their best (most-preferred) states. 
Where runs could be made to determine rankings with the group decision rules, 
once again only as much as a one-place change in ranking for a few applications 
of the group decision rules occurred (Table 6-4). Thus the results for Set 4 
(WORST/ 3 /MULT) and Set 5 (BEST/3/MULT) indicate that the assumptions made by 
the interviewees about other attribute states when assessing a utility 
function for an attribute did not significantly affect the rankings, at least 
at the group decision level of aggregation. 

Several variations on the nominal data were made to examine the effects 
of specific attributes on the rankings. The data for Set 6 (SAFETY) were 
derived from Set 1 (NOM/5/MULT) by fixing the attribute state for safety 
(Attribute #1) at 3 for all systems. The results are shown in Table 6-4. The 
results are essentially identical to the results for Set 1, thus eliminating 
the difference in safety as a sole factor in determining the rankings. The 
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in-core thermionic (ICT) rises 2 places because it is no longer penalized 
for a (highly weighted) safety rating of 6. The liquid cooled out-of-core 
thermionic (LOCTP) drops 2 places because its higher safety rating supported 
its somewhat lower score on the other attributes. With safety removed, it 
dropped in the rankings. 

The data for Set 7 (RADAREA) were derived from Set 1 (N0M/5/MULT) by 
fixing the attribute state for radiator area (Attribute #2) at 108m^ for all 
systems. The results are shown in Table 6-4. The heat-pipe Brayton (HBO) and 
heat-pipe thermophotovoltaic (HTPVP) rise 2 places since their large radiator 
areas no longer penalize them. The results are essentially identical to the 
results for Set 1, thus eliminating the difference in radiator area as the 
sole factor in determining the rankings. 

The data for Set 8 DESREL were derived from Set 1 (NOM/5/ MULT) by 
fixing the attribute state for design reliability (Attribute #3) at 2 for all 
systems. The results are shown in Table 6-4. The results are essentially 
identical to the results for Set 1, thus eliminating the difference in design 
reliability as the sole factor in determining the rankings. 

The data for Set 9 (TECHMAT) were derived from Set 1 (N0M/5/MULT) by 
fixing the attribute state for technical maturity (Attribute #4) at 3.8 for 
all systems. The results are shown in Table 6-4. The results are essentially 
identical to the results for Set 1, except the heat-pipe thermoelectrics (HTEP, 
HTEPa) reverse order and the liquid-metal cooled Stirling (LSH) drops 3 posi- 
tions because it loses its high advantage in technical maturity contribution. 

The data for Set 10 (FEASCOST) were derived from Set 1 (NOM/5/MULT) by 
fixing the attribute state for cost (Attribute #5) at $240 million for all 
systems. The results are shown in Table 6-4. The results are essentially 
identical to the results for Set 1, except for the liquid-metal Stirling (LSH) 
which drops three places because it loses its advantage of having a relatively 
low cost while the heat-pipe Brayton (HBO) rises in the rankings due to its 
penalty for a somewhat high cost. 

The data for Set 11 (SURVIV) were derived from Set 1 (NOM/5/MULT) by fix- 
ing the attribute state for survivability (Attribute #6) at 5 for all systems. 
The results are shown in Table 6-4. Little change is observed, even for the 
low survivability systems because they tend to have other low attribute values 
holding them down in the rankings. 

The data for Set 12 (DORMAN) were derived from Set 1 (NOM/5/MULT) by 
fixing the attribute state for dormancy (Attribute #7) at two for all systems. 
The results are shown in Table 6-4. The in-core thermionic (ICT) drops in 
position when dormancy is removed because its high score (a 10) is removed. 

The data for Set 13 (PRODUC) were derived from Set 1 (NOM/5 /MULT) by 
fixing the attribute state for producibility (Attribute #8) at three for all 
systems. The results are essentially identical to the results for Set 1, 
except for the liquid-metal cooled thermoelectric (LTEP) , which drops three 
positions in the ranking because it relies heavily on its high producibility 
for its position. 
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E. GENERAL CONCLUSIONS 

The general conclusions can be made from Table 6-4, which summarizes the 
results from the application of the group decision rules to the baseline data 
of Set 1 (N0M/5/MULT) . All other runs resulted in only minor variations on 
the rankings of Table 6-4. First and second rankings always went to heat-pipe 
thermoelectrics (HTEP, HTEPa) • The third place ranking usually went to the 
heat-pipe Stirling (HSH) , followed by fourth and fifth place with the in-core 
thermionic (ICT) and heat-pipe out-of-core thermionic (HOCTP). Sixth through 
tenth place went to the liquid-metal thermoelectric (LTEP) and Stirling (LSH), 
followed by the heat-pipe AMTEC (HAP), Brayton (HBO), and Rankine (HRL). The 
least preferred systems were the liquid-metal out-of-core thermionic (LOCTP) 
liquid-metal AMTEC (LAP), heat-pipe thermophotovoltaic (HTPVP) , liquid-metal 
Brayton (LBO) and Rankine (LRL), and the gas-cooled Brayton (GBH). 

Variations on the baseline data of Set 1 (N0M/5/MULT) , as made in data 
Set 2 through Set 5, made at most a two-place change in the rankings as deter- 
mined by the group decision rules, with typically no change. Data Set 6 and 
Set 13 fixed each of the attribute states and made changes in the ranking as 
compared to the baseline data of Set 1 (N0M/5/MULT) of as much as three places 
in ranking. Data Set 6 improved the preference for the in-core thermionic 
(ICT) by 2 places because of a lower safety rating on that system, which was 
not counted against it in Set 11. Data Set 7, where radiator area is dropped 
as an attribute, improved the heat-pipe Brayton and heat-pipe thermophoto- 
voltaic (HTPVP) by two places due to their large radiator areas. Data Set 9, 
where technical maturity is dropped, causes the liquid-metal Stirling (LSH) to 
drop three places due to its high reliance on technical maturity. The liquid- 
metal Stirling (LSH) also relies on low cost for a high ranking. When cost is 
eliminated as an attribute, as in data Set 10, the liquid-metal Stirling drops 
three places in the rankings. In data Set 13, where producibility is dropped, 
the liquid-metal cooled thermoelectric (LTEP) drops three places due to its 
reliance on high producibility in the scoring. 

It was not possible to rank the alternative systems on the basis of any 
one attribute. 

In summary, the top three rated systems were virtually unchanged with 
all twelve variations in assumptions across groups. The heat-pipe thermoelec- 
trics (HTEP, HTEPa) and heat-pipe Stirling (HSH) were the top three systems. 
Some shifting occurred in the fourth place although the in-core thermionic 
(ICT) tended to come up most often. 

The decision analysis process was viewed as useful for (1) reducing the 
large number of subsystem combinations to a manageable number; (2) character- 
izing and communicating the alternatives; and (3) providing the rationale and 
support for R&D planning to carry forward with the more promising technologies. 


F. CRITIQUE OF MULTIATTRIBUTE DECISION ANALYSIS METHODOLOGY 

The multiattribute decision analysis methodology was successful in all 
of the 11 interviews in ranking all the alternative system concepts. The 
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group decision rules were capable of aggregating preferences by groups, and, 
in general, the three group decision rules were in agreement. 

The multiattribute decision analysis was a deterministic analysis, as 
contrasted with a probabilistic analysis, and so did not completely reveal the 
technical experts' opinion as to the attribute states of the alternative sys- 
tems. A better analysis could have been undertaken if the attribute states 
had been estimated probabilistically. Either a discrete probability tree or 
a Monte Carlo simulation model, using subjectively estimated cumulative prob- 
ability distributions for the attribute states of the alternative systems, 
would have been sufficient data for a probabilistic analysis. The present 
analysis does not incorporate the uncertainties in the attribute state esti- 
mates. As a result, the process highlighted the need for more technical 
information about the systems and the degree of confidence to be placed on 
such information. 

The interview times could have been shortened if only three-point rather 
than five-point estimates had been made of the attribute utility functions. 

With the worst-state and the best-state used for two of the three points, 
questions for only one-attribute utility value need be asked in the interviews. 
Comparison of Tables 6-1 and Table 6-4 show that only minor differences in the 
rankings would have resulted in the group decision rules. Because continuity 
and monotonicity of preferences can be assumed for the attribute states, an 
attribute utility function of the "constant risk aversion" form: 

u(x) = a + be cx 

would have sufficed. Given the high premium for short interview times, it is 
recommended that in the future, unless there is strong reason to believe that 
the utility function is not represented by such a function with sufficient 
accuracy, the attribute utility functions be derived from three-point 
estimates. 

It was difficult for some, and impossible for others, of the interview- 
ees to assess gambles with respect to the set of attributes at their worst 
states. Had the system concepts been determined further in advance, the 
attribute worst states could have been made more desirable. It is highly 
recommended, in future multiattribute decision analyses, that the system 
states be determined before the interviews are conducted. This will also 
preclude the unfortunate situation in which the system states are ultimately 
determined to lie outside the range of the assessed attribute states. 

During the course of the process, the need for displaying the source 
calculations of the rankings was identified. Forays into piles of computer 
listings, no matter how comprehensive, were deemed insufficient. Although 
such transparency can be shown to some extent with summary graphics, an 
interactive version of the model to allow display of both intermediate and 
summary calculations is needed. 
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SECTION VII 


CONCORDANCE OF RANKINGS 


A. INTRODUCTION 

This section presents and discusses the results of concordance calcula- 
tions for the rankings presented in Section VI (see subsection C in Section II 
for a discussion of concordance statistics). Two different types of concor- 
dances were calculated and analyzed: 

(1) Individual rankings within groups. 

(2) Group rankings with different group decision rules. 

The purpose of these concordance calculations and analyses was to ascertain 
how robust, or conversely, how sensitive the rankings were to: differences 
among individuals within groups and differences in group decision rules. In 
general, the rankings were highly concordant across individuals within groups 
and across different group decision rules, implying that the rankings pre- 
sented in Section VI were indeed robust. 

The concordance of the rankings given to the sixteen alternative system 
concepts by individuals within the three groups was examined in two ways: 

(1) by comparing the individual rankings within groups (Table 7-1) for each 
of the 40 runs previously described in Section VI; and (2) by comparing the 
group rankings according to the additive, Nash, and rank sum rules (Table 7-2). 
The following observations can be made: 

(1) There is not, of course, perfect agreement throughout the ranking. 

(2) There are several instances in which the ranks assigned to several 
alternatives by one interviewee in a group seem to be at variance 
from those given by the other interviewees in the group. 

(3) The concordance measures in every instance, however, are highly 
significant. Each of them is significant well below the 1% level; 
many are significant below the 0.1% level. 

(4) Accordingly, by each of the comparison methods, there is substan- 
tial agreement as to the rankings of the sixteen alternative 
systems within each of the three groups of interviewees. 

The chi-square values corresponding to the coefficients of concordance 
indicate no instances in which there is no significance at a minimum 1% level. 
The other 78 chi-square values are significant well beyond the 5% level. This 
indicates excellent agreement among interviewees and among group decision 
rules. Although it is possible to examine concordance among different methods, 
this was not done, due to time constraints. The majority of lower concordance 
values occurred within Group 2 - Systems Definition due to different weightings 
of certain attributes which have moderate impacts (cost, design reliability, 
survivability, producibility) on ranking. 
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Table 7-1. Summary of Kendall's Coefficient of Concordance (W) and the 
Associated Chi-Square Values (X^) for all Runs: Individuals 

Within Groups 





i 


2 


3 

RUNS 

METHOD 

W 

CM 

w 

X 2 

w 

x 2 


1-3 

N0M/5/MULT 

0.9596 

57.57 

0.9314 

41.91 

0.9072 

40.82 

5-7 

N0M/5/LIN 

0.9750 

58.50 

0.9366 

42.15 

0.9784 

44.03 

9-11 

N0M/3/MULT 

0.9647 

57.88 

0.9294 

41.82 

0.9327 

41.97 

17-19 

WORST/ 3 /MULT 

0.9588 

43.15 

0.8725 

39.26 

0.9046 

40.71 

21-23 

BE ST/ 3 /MULT 

0.9647 

57.88 

0.9268 

41.71 

0.9242 

41.59 

25-27 

N0M/5/MULT 
SAFETY @ 3 

0.9665 

57.99 

0.9647 

43.41 

0.9784 

44.03 

29-31 

N0M/5/MULT 
RAD AREA @ 108 

0.9732 

58.39 

0.9163 

41.24 

0.9092 

40.91 

33-35 

N0M/5/MULT 
DES REL @ 2 

0.9449 

56.69 

0.8556 

38.50 

0.8908 

40.09 

37-39 

N0M/5/MULT 
TECH MT @ 3.8 

0.9246 

55.48 

0.9144 

41.15 

0.8974 

40.38 

41-43 

N0M/5/MULT 
COST @ $240M 

0.9621 

57.73 

0.8791 

39.56 

0.9131 

41.09 

45-47 

N0M/5/MULT 
SURV @ 5 

0.9441 

56.65 

0.8693 

39.12 

0.9137 

41.12 

49-51 

N0M/5/MULT 
DORM @ 2 

0.9511 

57.07 

0.9346 

42.06 

0.9399 

42.29 

53-55 

N0M/5/MULT 
PROD @ 3 

0.9408 

56.45 

0.8876 

39.94 

0.9229 

41.53 
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Table 7-2. Summary of Kendall's Coefficient of Concordance (W) and the 
Associated Chi-Square Values (X^) for all Runs: Group 

Decision Rules 




] 

1 

2 


3 


RUNS 

METHOD 

W 

x 2 

w 

x 2 

w 

X 2 

1-3 

NOM/ 5 /MULT 

0.9987 

44.94 

0.9902 

44.56 

0.9869 

44.41 

5-7 

NOM/ 5/LIN 

0.9984 

44.93 

0.9964 

44.84 

0.9971 

44.87 

9-11 

N0M/3/MULT 

0.9987 

44.94 

0.9895 

44.53 

0.9889 

44.50 

17-19 

WORST/ 3/MULT 

0.9922 

44.65 

0.9738 

43.82 

0.9926 

44.67 

21-23 

BEST/ 3 /MULT 

0.9980 

44.91 

0.9833 

44.25 

0.9816 

44.17 

25-27 

NOM/ 5 /MULT 
SAFETY 0 3 

0.9974 

44.88 

0.9971 

44.87 

0.9969 

44.86 

29-31 

NOM/ 5 /MULT 
RAD AREA 0 108 

0.9984 

44.93 

0.9915 

44.62 

0.9863 

44.38 

33-35 

NOM/ 5 /MULT 
DES REL @ 1 

0.9948 

44.76 

0.9948 

44.76 

0.9862 

44.38 

41-43 

NOM/ 5 /MULT 
COSTS 0 $240M 

0.9935 

44.71 

0.9971 

44.87 

0.9817 

44.18 

45-47 

NOM/ 5 /MULT 
SURV 0 5 

0.9944 

44.75 

0.9739 

43.82 

0.9758 

43.91 

49-51 

NOM/ 5 /MULT 
DORM 0 2 

0.9967 

44.85 

0.9843 

44.29 

0.9912 

44.60 

53-55 

NOM/ 5 /MULT 
PROD 0 3 

0.9987 

44.94 

0.9774 

43.98 

0.9859 

44.37 
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APPENDIX A 

QUESTIONNAIRE USED FOR INTERVIEWS 
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SAFETY 


Best 

Case 


Worst 

Case 


firgmiflZP re- flight Launch/ flight 


10 


safety level exceeds normal 
pre-flight precautions due 
to nature of power system 


safety exceeds 
current launch/ 
flight standards 
—remains subcrit. 
if immersed in 
water 


7 


low risk of exposure in 
of pre- flight accident; 
safety level comparable 
current practice 


event low risk of 

problems in event 
to of abort— remains 
subcritical if 
immersed in water 


3 some risk of radiation some risk of prob- 

exposure in event of pre- Ians in event of 
flight accident abort 


0 pre-flight accident 

could lead to criticality 
and/or radiation dosage 
to personnel 


plausible scenarios 
for criticality/ 
explosion in event 
of water immersion 
f very large objects 
reaching the ground 
or loss of coolant 
after deployment 


Post-mission 


safety exceeds 
current disposal 
practices/core will 
disperse safely in 
event of reentry 


low risk of large 
objects reaching 
the ground 


some risk of large 
objects reaching 
the ground 


large radioactive 
objects reach the 
ground 


DESIGN RELIABILITY 


Best 

Case 


Worst 

Case 


10 Very high— -tolerant to single point and multiple 
failures with graceful degradation In performance. 


8 High— tolerant to single point failures with 
graceful degradation in performance. Tolerant 
within certain limits to multiple-point failures. 


6 Moderate- A: tolerant to single point failures with 
graceful degradation in performance— low risk of 
system failure. Moderate tolerance to multiple 
failures. 


4 Moderate-B: more limited tolerance to single point 
failures with more dramatic degradation in system 
performance over time— some risk of system 
failure. Low tolerance to multiple failures 


2 Moderate-C: lower tolerance to single point 

failures and very low tolerance to multiple 
failures with moderate risk of system failure. 

0 Low— susceptable to single point failures which 
propagate into overall system failure (through 
loss of coolant or damage to control system). 
Similarly* multiple failures result in system 
failure. 
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TECHNOLOGICAL MATURITY 


Best 

Case 


Worst 

Case 


10 Maximum technological maturity requiring a minimum 
of new developments. 


8 Advanced technological maturity requiring some 
minor developments in particular subsystems. 


6 Moderate technological maturity requiring some 
major developments in minor subsystems. 

4 Some technological maturity requiring significant 
developments in minor subsystems. 

2 Low technological maturity requiring significant 
developments in major subsystems. 


0 Virtually no technological maturity requiring full 
scale technology developments in major subsystems. 



SURVIVABILITY 


Best 


Worst 


10 Very high likelihood of surviving military threats 
at required levels and higher without loss of 
performance. Also a high likelihood of surviving 
meteorite impacts without loss of performance. 


7 Likely that system will survive designed levels of 
military threat and meteorite hazard without loss 
of performance. 

3 Moderate likelihood of surviving designed levels 
of military threat and meteorite hazard without 
loss of performance. 

0 Low likelihood of surviving military threats at 
design levels. High risk of system failure in 
event of meteorite impact or military induced 
damage. 
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tSAB -F9fcL0WINQ/DORMANCY CAPABILITY 


Best 


Worst 

Case 


10 Multiple restart capability throughout mission 
lifetime with high degree of load following 
capability using electrical switching to follow 
load closely in steps and responding quickly to 
rapid drops in load. Minimal power storage 
requirements for startup enabling long periods of 
dormancy. 


7 Multiple restart capability throughout mission 
lifetime. Moderate power storage required for 
restarts due to power requirements thus dormancy 
period is shorter than in best case. Moderate 
load following capability due to gas valving 
system for dumping excess energy. 


3 Multiple restart capability throughout mission 
life. High power requirements for startup. 
Reduced load following capability due to vapor 
valving for dumping excess energy. 


0 Poor load following capability-system runs at 
full power and vents excess heat using an unvalved 
system or one with electric shunt. No dormancy 
capability other than launch period prior to 
Initial start and no ability to shutdown after 
startup. 
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PR ODD CIBILITY / PRACTICALITY 


Best 

Case 


Worst 

Case 


10 Highly modularized independent subsystems with 
simple interfaces. Easily manufactured materials, 
parts, and assemblies and no extraordinary 
tooling/facilities required. All components can 
be pre-flight tested independently without 
assembling the whole system. 


7 System is somewhat modular with some complex 
interfaces. The fabricability is similar to other 
spaceborne systems with comparable fabrication 
problems. The subsystems are, for the most part, 
testable independently. 


3 Minimal modularity with complex Interfaces between 
most of the subsystems. The fabricability is more 
difficult than comparable spaceborne systems 
requiring some special materials. Some of the 
subsystems are difficult to test without a vacuum 
environment. 


0 Virtually no modularity-system is an Integrated 
whole with complex Interfaces. It is very 
difficult to manufacture since special materials, 
tooling, and facilities are required. Major 
subsystems are not testable and may require space 
testing to determine flight worthiness. 
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ATTRIBUTE: SAFETY 
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IF YOU KNEW THAT ALL OTHER ATTRIBUTES WERE AT 
THEIR BEST STATES? 

INDIFFERENCE POINT 


ATTRIBUTE: safety 
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ATTRIBUTE: SAFETY 
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ATTRIBUTE: radiator area 



=>i 

>1 
lu =; 

CC fe 

< < 


o 


o 


=r X 


LU 


LU 


LU 

CC 

LU 

£ 

CO 

LU 

I- 

X 

£ 

CC 


co*> 

* * 

I! 

u_ 2 

oa 

< m 
>“ 

izT' 

0 LU UJ 
= CC -I 

1 LU CD 

§r 

LU _ 


CC 

LU 

X 


I- Ij 
2 < 

O *- 
0- < 

LU X 
O H 

5 £ 

CC LU 
LU Z 
LU V 
LU 


O 


P^. 

CO 

LU 

I- 

< 

H 

CO 


CC 

O 


Q 

^ >- 


o£ 


o 

o. 

LU 

O 


1“ 

CO LU 


CC 

LU 


U> Ua 

t ” > LU 


LU ^ 


SfcP 


A-15 


IF YOU KNEW THAT ALL OTHER ATTRIBUTES WERE AT 
THEIR BEST STATES? 

INDIFFERENCE POINT 


ATTRIBUTE: RADIATOR AREA 
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ATTRIBUTE: RADIATOR AREA 
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ATTRIBUTE: design reliability 
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IF YOU KNEW THAT ALL OTHER ATTRIBUTES WERE AT 
THEIR BEST STATES? 

INDIFFERENCE POINT 


ATTRIBUTE: design reliability 





m 

LL 2 

O m 

w jjj 





A-19 


ATTRIBUTE: design reliability 



cc 

o 





HI 

X 

I- 


HI 

HI 


5 

ts 

CD 


H 


Hl 

cc 

HI 

LL 


Q 



r.. 

h 

CD 


< 

CD 

% 


* 


I- 

Z 

5 

CL 

UJ 

O 

z 


Hi 

CC 

HI 

LL 

LL 


O 


A-20 


ATTRIBUTE: TECHNICAL MATURITY 
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IF YOU KNEW THAT ALL OTHER ATTRIBUTES WERE AT 
THEIR BEST STATES? 


ATTRIBUTE: technical maturity 
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ATTRIBUTE: technical maturity 
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ATTRIBUTE: estimated cost to reach technical feasibility 
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IF YOU KNEW THAT ALL OTHER ATTRIBUTES WERE AT 
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ESTIMATED COST TO REACH TECHNICAL FEASIBILITY 
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ATTRIBUTE: SURVIVABILITY 
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ATTRIBUTE: survivability 


Ol 



cc 

O 





oB 

“I 

3 H 

< £ 

> “ 

mm * 

I Z : 

O UJ iu 

CC hJ 




A-28 


ATTRIBUTE: SURVIVABILITY 
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IF YOU KNEW THAT ALL OTHER ATTRIBUTES WERE AT 
THEIR BEST STATES? 

INDIFFERENCE POINT 


ATTRIBUTE: lsab- F ettswiNG/roRMANCY capability 
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ATTRIBUTE: L0AB -FOLLOW ING/TORMANCY CAPABILITY 


LU 






O “J 

w jjj 

>“ 

I lZ r^* 

* 

U LU LL1 
= cn -j 
“ LU CD 

o§ « 

ll = : 



A-32 



ATTRIBUTE: PRDDUCIBILITY 
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IF YOU KNEW THAT ALL OTHER ATTRIBUTES WERE AT 
THEIR BEST STATES? 


ATTRIBUTE: PRODUCIBILITY 
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ORDER OF IMPORTANCE OF ATTRIBUTES. 
WHICH ATTRIBUTE WOULD YOU CHANGE FROM ITS 
WORST STATE TO ITS BEST STATE? 
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IMPORTANCE OF DESIGN RELIABILITY 
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APPENDIX B 


FLOW CHART OF INFORMATION FLOWS AND MATEUS PROGRAM 
USED TO CALCULATE RANKINGS WITH SAMPLE RUNS 
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The MATEUS Computer Program 


I. MATEOS Computer Program Overview 

The Computer Program MATEUS: Multi ATtrlbute fiyalution of JLtilitie.2. was 
used to process the Interview data and to determine ordinal and cardinal 
rankings of the alternative systems for individual and group preferences. 
MATEUS is presently written in Microsoft FORTRAN-80, Release 3.4, December 
1980, and will run on any Microsoft FORTRAN-80 compatible microcomputers (with 
8080, 8085, Z80, 8086, and 8088 microprocessors). Microsoft FORTRAN-80 is 
essentially equivalent to FORTRAN IV, and with only minor modification 
(principally in the READ and WRITE statements) should run on any computer with 
a FORTRAN processor. (See also Figure B-l.) 

MATEUS does three major computations: (1) It calculates the 
multiattribute utilities of outcomes based on the Keeney Multiplicative Model 
for multiattribute decision analysis, (2) it calculates utilities and rankings 
of alternative systems based on the the multiattribute utilities of outcomes 
and a discrete probability tree, and (3) it calculates group preferences 
corresponding to three group decision rules. 

MATEUS comprises ten modules, each module partitioned into lower-level 
modules. Figure B-2 is a Program Tier Chart for MATEUS. The number above the 
upper right corner of a module gives the calling module. A number preceded by 
an "S" above the upper left corner of a module indicates that the module is a 
subroutine called at more than one place in the Program. Module SI is called 
by Module 1.8 and Module 1.11. Figure B-3 is a Top-Level Program Flowchart 
for MATEUS. The top-level flowchart has a DO-loop that processes the data for 
each individual. Table B-1 gives the principal variable, array, and array 
index definitions for MATEUS. 

A. Module MAIN 

Module "MAIN" is the Main Routine for MATEUS and is the calling routine 
for all other routines. It initializes the dimensions of all arrays, and 
contains the structure of the DO-loop that processes the data for each 
individual. 

B. Module DATA1 

Module "DATA1" is called by the MAIN Module. It inputs the data for the 
probabilities of the decision tree and the attribute states for all outcomes. 
In the application of MATEUS to the evaluation and ranking of Electric and 
Hybrid Vehicles, no probabilistic analysis was undertaken, so that all 
probability nodes had only one path emanating from them, each with an 
associated probability of 1.0. 

c. Module JAIA2. 

Module "DATA2" is called by the MAIN Module. It Inputs the data for the 
calculations for each individual. The data comprises the attribute scaling 
constants and (x,y) pairs of data points for piece-wise linear fits to the 
attribute utility functions. In the application of MATEUS to the evaluation 
and ranking of Electric and Hybrid Vehicles, both three-point and five-point 
fits to the attribute utility functions were used. 
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INTERVIEWS 


Figure B 


i 

DATA CODING 
FORMS 


PMAIN 


DECLARES ARRAY SIZES, CALLS 
ALL SUBROUTINES 


t 



READS ATTRIBUTE DATA, UTILITY 
FUNCTION PARAMETERS, PERFORMANCE 
AND WORKLOAD PROBABILITIES 


CALCULATES MASTER SCALING CONSTANT 
OF MULTIATTR IBUTE UTILITY FUNCTIONS 


CALCULATES UTILITIES OF EACH ATTRIBUTE 
FOR EACH PERFORMANCE-WORKLOAD SCENARIO 
AND FOR EACH INDIVIDUAL 


OOHB1NES ATTRIBUTE UTILITIES INTO 
OUTCOME UTILITIES FOR EACH PERFORMANCE- 
WORKLOAD SCENARIO AND EACH INDIVIDUAL 


MANIPULATES PROBABILITIES ASSIGNED TO 
PERFORMANCE WHEN PERFORMING SENSITIVITY 
ANALYSIS (FRACTIONAL FACTORIAL DESIGN) 


t 


CALUS 


COMBINES OUTCOME UTILITIES, USING 
PERFORMANCE AND WORKLOAD PROBABILITIES, 
INTO SYSTEM UTILITIES FOR EACH 
INDIVIDUAL, ASSIGNS INDIVIDUAL RANKS 
TO SYSTEMS 


t COMPUTES CROUP RANKS USING 



1. Information Flows and Relevant Computer Programs 
Used to Rank Alternatives 


B-4 


















1.4 



TIER 1 TIER 2 


Figure B-2. MATEUS Program Tier Chart 
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Figure B-3. MATEUS Top-Level Program Flowchart 
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The MATEUS Computer Program 


D. Module CALXKM 

Module "CALXKM" is called by the MAIN module. It calculates the master 
scaling constant for the Keeney Multiplicative Formulation of the 
multiattribute decision analysis model, given the attribute scaling constants 
of an individual. It has been proven by Keeney that the master scaling 
constant must be greater than -1.0, and that for the sum of the scaling 
constants less than 1.0, the master scaling constant must be greater than 0.0, 
for the sum of the scaling constants equal to one the Multiplicative 
Formulation is replaced by a Linear Formulation, and that for the sum of the 
scaling constants greater than 1.0, the master scaling constant must be less 
than 0.0. This information is used to determine a starting point for a 
Newton-Raphson iteration for the master scaling constant. 

E. Module SALMI 

Module CALUAT is called by the MAIN Module. It calculates the attribute 
utility function values for the attributes of each outcome given the outcome 
states and the individual utility functions. 

F. Module CALUO 

Module CALUO is called by the MAIN Module. It calculates the outcome 
utility function values for an individual for the outcomes, given the outcome 
attribute utility function values, and the individual master and attribute 
scaling constants. 

G. KPdulg-.CALIiS 

Module CALUS is called by the MAIN Module. It calculates the alternative 
(system) utility function value for each alternative given the outcome utility 
function values and the probabilities of the decision tree. In the 
application of MATEUS to the evaluation and ranking of Electric and Hybrid 
Vehicles, the analysis was deterministic and the decision tree defaulted to 
probability nodes with one path emanating with probability 1.0. 

F. Module GROUP 

Module GROUP is called by the MAIN Module. It calculates the group 
decision rule values for three group decision rules: (1) The Additive Rule, 
(2) the Nash Bargaining Rule, and (3) the Rank Sum Rule. 

G. Module CONCRD 

Module CONCRD is called twice by the MAIN Module— once for the individual 
rankings and once for the three group decision rules. It calculates Kendall's 
Coefficient of Concordance. 

H. me MMLESB. 

File ARRAY.FOR is "INCLUDED at compile time for the MAIN Routine. It 
allocates memory space for all arrays with FORTRAN "DIMENSION" AND "DOUBLE 
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The MATEUS Computer Program 


PRECISION" statements. It is the only file that has to be modified to change 
the maxi mum size of arrays. 

////////////////////////////////////////////////////////////////////////////// 
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IA 

IC 

ID 

IG 

IP 

IS 

IW 


Index for Attributes. IA = 1,...,NIA. 

Index for Coefficients of piece-wise linear fit to Attribute 
Utility Data. IC = 1 NIC. 

Index for Attribute Utility Data. ID = 1,...,NID. 

Index for Group Decision Rules. IG = 1,...,3. 

Index for Probability Node Paths. IP = 1,...,NIP. 

Index for Systems or Alternatives. IS = 1,...,NIS. 

Index for Kendall* s Coefficient of Concordance Group. 
IW si ,2 


JRUNID Integer Identifier for the Input and Output Data of the 

MATEUS run. 

JDIAG Integer Level of Diagnostic display during run. (Not 

implemented in Version 1.1.) 


NIA 

Number 

of 

NIC 

Number 

of 

NID 

Number 

of 

NIP 

Number 

of 

NIS 

Number 

of 


Attributes. 

Coefficients. NIC = NID - 2. 
Attribute Data Values. 
Probability Paths. 

Systems or Alternatives. 


AT(IS,IP,IA) 

CHISQR(IW) 

IRS(II,IS) 

IRG(IG.IS) 

JDF(IW) 

PROB(IS,IP) 

RG ( IG , IS ) 

RS(II,IS) 

uat(is,ip,ia) 

UATC(IA, IC) 
UATD(IA,ID) 
UO(II,IS,IP) 

usdi.is) 

VG(IG,IS) 

XK(II,IA) 

XKM(II) 


Attribute State for Attribute IA of the Outcome of 
Probability Path IP for System IS. 

CHI-Square Statistic for Group IW. 

Array RS(II,IS) converted from real to integer data type. 
Array RG(II,IS) converted from real to integer data type. 
Degrees of Freedom for Group IW. 

Probability assigned to Probability Path IP for System IS. 
Preference Rank assigned to System IS by Group Decision Rule 
IG . 

Preference Rank assigned to System IS by Individual II. 
Utility assigned to Attribute IA for Outcome of Probability 
Path IP (by some Individual II). 

Utility Coefficent IC of piece-wise linear fit to Utility of 
Attribute IA (for some Individual II). 

Utility Data ID for Attribute IA (assessed by some Individual 
II). 

Utility assigned to Outcome of Probability Path IP for System 
IS by Individual II. 

Utility assigned to System IS by Individual II. 

Value assigned to System IS by Group Decision Rule IG. 

Scaling Constant assigned to Attribute IA by Individual II. 
Master Scaling Constant for Individual II. 


Table B-1. Principal Variable, Array, and Array Index Definitions for HATEUS. 
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File: B: MAIN. FOR 


C PROGRAM MATEUS 


C 

C THIS IS THE PROGRAM MATEOS: MULTI ATTRIBUTE EVALUATION OF UTILITIES. 

C IT IS WRITTEN IN MICROSOFT FORTRAN- 80, RELEASE 3- 1 *, DECEMBER 1980. 

C IN GENERAL, FORTRAN PROCESSORS WILL REQUIRE THAT MINOR MODIFICATIONS 
C BE MADE TO THE PROGRAM. 

C 

C PROGRAMMER: R. F. MILES, JR. 

C JET PROPULSION LABORATORY 

C PASADENA, CALIFORNIA 91009 

C 

C VERSION: 1.1x1 7/21/83. 

CiHHiHHHimHiiHmiiimimmmHmimmmfmmmHC 

C THIS IS THE MAIN ROUTINE OF THE PROGRAM MATEUS. 

C THIS IS THE MAIN ROUTINE. IT DECLARES THE SIZES OF THE ARRAYS. IT 
C IS THE MAIN CALLING ROUTINE FOR ALL SUBROUTINES. IT CONTAINS THE DO 
C LOOP FOR INDIVIDUALS, II a 1,...,NII. IT ENDS THE PROGRAM. 

C C 

C**«C DIMENSION THE ARRAYS. {MODULE 1} 

INCLUDE B: ARRAY. FOR 

C C 

C*»»c INITIALIZATION. {MODULE 2} 

C 

WRITE (5,100) 

100 FORMAT (» START MAIN ROUTINE’) 

C 

READ (7,110) JRUNID , JDIAG , Nil , NIS , NIP , NIA , NID 
110 FORMAT (//IX, 7110) 

C 

NIC = NID - 2 

C C 

C«»»C WRITE TITLE TO TERMINAL AND DISK. {MODULE 3) 

C 

C TERMINAL (JUNIT a 5) AND DISK (JUNIT a 8 FOR FILE F0RT08.DAT) . 

C 

DO 150 IU=1 ,2 

C INSERT W 20H’’ FOR PRINTER AND "OAK" FOR FILE. 

IF (IU .EQ. 1) LF a Z’20 * 

IF (IU .EQ. 1) JUNIT a 5 

IF (IU .EQ. 2) LF a Z’OA’ 

IF (IU .EQ. 2) JUNIT a 8 

C 

C ••• PROGRAM TITLE ••• 

WRITE (JUNIT, 120) 

120 FORMAT (IX, 35X, ’MATEUS’) 

WRITE (JUNIT, 130) LF,LF 

130 FORMAT (A1,20X,* MULTI ATTRIBUTE EVALUATION OF UTILITIES ’ /A1 ) 

C 

c *** PROGRAM INITIALIZATION PARAMETERS ••• 

WRITE (JUNIT, 140) LF,LF, JRUNID, LF, JDIAG, LF, Nil, LF, NIS, LF.NIP, 
1 LF,NIA,LF,NID,LF 

140 FORMAT (A1/A1,’RUN IDENTIFICATION NUMBER: ',15,/ 

1 A1 , ’DIAGNOSTIC DISPLAY LEVEL: ',15/ 
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File: B:MAIN.FOR 


2 A1,» NUMBER OF INDIVIDUALS: ',15/ 

3 A1,» NUMBER OF SYST01S: ‘,15/ 

4 A1,' NUMBER OF PROBABILITY PATHS: ',15/ 

5 A1,' NUMBER OF ATTRIBUTES: ',15/ 

6 A1,' NUMBER OF ATTRIBUTE DATA: ',15/AI ) 


150 CONTINUE 
C 

C ••• DELAY TO READ TERMINAL 
DO 160 1=1,5000 

DELAY = DELAY + 1.0 
160 CONTINUE 


C C 

C««»C INPUT DATA FOR PROBABILITIES AND ATTRIBUTE STATES. {MODULE 4} 

C 

CALL DATA1 (PROB, AT,NIS,NIP, NIA) 

c c 


C»»*C DO LOOP FOR INDIVIDUAL CALCULATIONS. {MODULE 5) 
C 

DO 180 11=1, Nil 
WRITE (5,170) II 


170 FORMAT (/IX, 'CALCULATIONS FOR INDIVIDUAL ',13) 

C C 


C«»«C INPUT DATA FOR SCALING CONSTANTS AND ATTRIBUTE UTILITY 
C«»«C FUNCTIONS FOR INDIVIDUAL II. {MODULE 6} 

C 

CALL DATA2 ( XK , UATD , II , Nil , NIA , NID ) 

C C 

C»»»C CALL SUBROUTINES FOR INDIVIDUAL II CALCULATIONS. {MODULE 7} 

C 

CALL CALXKM(XK,XKM,II,NII,NIA) 

CALL CALUAT( AT, UATD , UATC , UAT, II , NIS , NIP , NIA , NID , NIC ) 

CALL CALUO( XK , XKM , UAT, UO , II , Nil , NIS , NIP , NIA ) 

CALL CALUS( PROB , UO , US , RS, IRS, II , Nil , NIS , NIP ) 

C C 

C»«»C END DO LOOP FOR INDIVIDUAL CALCULATIONS. {MODULE 5) 

C 

1 80 CONTINUE 

C C 

C»««C CALCULATE INDIVIDUAL CONCORDANCE. {MODULE 8} 

C 

CALL CONCRD(RS,SRX, TIES, STIES, W, CHISQR, JDF,1 , Nil, NIS) 

C C 

C«*»C END CALCULATIONS FOR INDIVIDUALS. {MODULE 9) 

C 

WRITE (5,200) 

200 FORMAT (/IX, 'END CALCULATIONS FOR INDIVIDUALS'/) 

C C 

C»»*C CALCULATIONS FOR GROUP RULES. {MODULE 10} 

C 

CALL GROUP( US , RS, IRS, VG, RG , IRG , Nil , NIS) 

C C 

C««»C CALCULATE GROUP CONCORDANCE. {MODULE 11} 

C 

CALL CONCRD ( RG , SRX , TIES , STIES , W , CHISQR , JDF , 2 , 3 , NIS ) 

C C 
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File: B: MAIN. FOR 


C»««C WRITE OUTPUT TO TERMINAL AND DISK. {MODULE 12} 

C 

CALL OUTPUT ( US , IRS , VG , IRG , W, CHISQR , JDF , Nil , NIS ) 

C C 

C»»»C END PROGRAM MATEUS. {MODULE 13} 

C 

WRITE (5,999) 

999 FORMAT (///IX, » EXIT MAIN PROGRAM'/) 

C 

STOP MATEUS 
C 

END 

CHmmmiimiimmmmHHimHiiimHHimmmHHiific 
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File: B:DATA1.F0R 


CHHiiHHHiHiitfHHimmmimmimHmtHmHtHHHHHc 

C SUBROUTINE DATA1 9/5/82 

CHHHHHHHHfHmmimfmiiimiHmtHmHHHmmHmc 

C SUBROUTINE DATA1 INPUTS THE DATA FOR PROBABILITIES AND ATTRIBUTE 
C STATES. 

CHHHHIlHHHflHttmilltlHIIIHHHHIIIIIHIHIIIIIIHIimMC 

SUBROUTINE D ATA1 ( PROB , AT , NIS , NIP , NI A ) 

C»«»C INITIALIZE. {MODULE 1} 

C 

DIMENSION PROB(NIS, NIP) ,AT(NIS,NIP, NIA) 

WRITE (5,100) 

100 FORMAT (/' ENTER SUBROUTINE DATA1 • ) 

C C 

C«»*C READ A WRITE PROBABILITY DATA PROB(NIS,NIP) . {MODULE 2} 

C 

DO 150 IS=1,NIS 
READ (7,110) 

110 FORMAT (IX) 

READ (7,120) (PROB(IS, IP), IP=1, NIP) 

120 FORMAT ( IX, 1 0F7.4 ) 

WRITE (5,130) IS 

130 FORMAT (/IX, 'PROBABILITY DATA FOR OUTCOMES (IS, IP) OF SYSTEM', 

1 ' (IS =',13,')') 

WRITE (5,140) ( PROB(IS, IP ),IPr1 ,NIP) 

140 FORMAT (1X,10F7.4) 

150 CONTINUE 

C C 

C»«»C READ & WRITE ATTRIBUTE DATA AT(IS,IP,IA) FOR OUTCOMES (IS, IP). 

C {MODULE 3) 

C 

DO 200 IS=1 , NIS 
DO 200 IP= 1, NIP 
READ (7,160) 

160 FORMAT (IX) 

READ (7,170) (AT(IS,IP,IA),IAs1 ,NIA) 

170 FORMAT (5(1X,E14.4)) 

WRITE (5,180) IS, IP 

180 FORMAT (IX, 'ATTRIBUTE DATA FOR OUTCOME 

1 '(IS s' ,13, • , IP s', 13,')') 

WRITE (5,190) (AT(IS,IP,IA) ,IA=1 ,NIA) 

190 FORMAT (5( IX, 1 PE14 .4) ) 

200 CONTINUE 

C C 

C««»C EXIT SUBROUTINE DATA1 . {MODULE 4} 

C 

WRITE (5,999) 

999 FORMAT (' EXIT SUBROUTINE DATA1 ' ) 

C 

RETURN 
C 

END 
C 


C 
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If ••*•«••••• tft 

c SUBROUTINE DATA2 8 / 28/82 

CHimiHHIfHHHtHHtmHIHmHHHmmilHHIHHHHHItlC 

C SUBROUTINE DATA2 INPUTS THE DATA FOR THE CALCULATIONS FOR EACH 
C INDIVIDUAL II. 

SUBROUTINE DATA2 ( XK , UATD , II , Nil , NIA , NID ) 

C«««C INITIALIZE. {MODULE 1} 

DIMENSION XK ( Nil , NIA ) , UATD ( NIA , NID ) 

WRITE (5,100) 

100 FORMAT (/» ENTER SUBROUTINE DATA2 ' ) 

C C 

C»»»C READ DATA FOR INDIVIDUAL (II). {MODULE 2} 

C READ SCALING CONSTANTS XK(II,IA) 

READ (7,105) 

105 FORMAT (/IX) 

READ (7,110) (XK(II,IA) ,IAs1 ,NIA) 

110 FORMAT (1X,10F7.4) 

WRITE (5,120) (XK(II,IA) ,IA=1 , NIA) 

120 FORMAT (IX, 'SCALING CONSTANTS XK(II,IA)' 

1 /( IX, 1 0F7.4) ) 

C READ ATTRIBUTE UTILITY DATA UATD(IA,ID) FOR INDIVIDUAL (II). 

READ (7,130) 

130 FORMAT (IX) 

WRITE (5,140) 

140 FORMAT (IX, 'ATTRIBUTE UTILITY DATA UATD (IA, ID) ' ) 

DO 180 IA=1 , NIA 
READ (7,145) 

145 FORMAT (IX) 

READ (7,150) (UATD(IA,ID) ,ID=1 ,NID) 

150 FORMAT (3( 1X,E16.4 ,F7.4) ) 

WRITE (5,160) IA 

160 FORMAT ( IX, 'ATTRIBUTE ',13) 

WRITE (5,170) (UATD(IA,ID) ,ID=1 ,NID) 

170 FORMAT (3(1X,1PE16.4,0PF7.4)) 

1 80 CONTINUE 

C C 

C«»«C EXIT SUBROUTINE DATA2. {MODULE 99) 

WRITE (5,999) 

999 FORMAT (' EXIT SUBROUTINE DATA2 ' ) 

C 

RETURN 

C 

END 

cmHiHHiHimiHimumiiimiHHiimmiiimimmiimtic 
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C SUBROUTINE CALXKM 2/6/83 

CHHmHKHmiHHfHfHHiiiHiHmHHmimHimifmtmfic 

C SUBROUTINE CALXKM CALCULATES THE MASTER SCALING CONSTANT XKM(II). 
SUBROUTINE CALXKM ( XK , XKM , II , Nil , NIA ) 


C*»«C INITIALIZE. {MODULE 1} 

C 

DOUBLE PRECISION XKM(NII) ,FG,G,DG,XKML 
DIMENSION XK(NII t NIA) 

WRITE (5,100) 

100 FORMAT (/• ENTER SUBROUTINE CALXKM*) 

C 

C ••• WRITE SCALING CONSTANTS XK(NII,NIA) ••• 

WRITE (5,110) II 

110 FORMAT (/IX, 'SCALING CONSTANTS XK( II, IA) FOR INDIVIDUAL (IIs», 

1 13,’)*) 

WRITE (5,120) (XK(II,IA),IAs1,NIA) 

120 FORMAT (1X.10F7.4) 

c 

C*«*C TEST FOR SIGN OF XKM(II). {MODULE 2} 

C 

SXK s 0.0 
DO 130 IA=1,NIA 

SXK s SXK+XK(II,IA) 

130 CONTINUE 
C 

WRITE (5,1 -40 ) II, SXK 

140 FORMAT (IX, » SUM OF ATTRIBUTE SCALING CONSTANTS FOR INDIVIDUAL ’, 
1 '(IIs',13,*) IS: ’ ,F8.4) 


C 

IF (ABS(SXK-I.O) .LT. 1 .OE-3) GO TO 160 
IF (SXK .GT. 1.0) GO TO 150 
IF (SXK .LT. 1.0) GO TO 170 

C C 

C««*C INITIALIZE XKM(II). {MODULE 3) 

C 

C ••• INITIAL XKM < 0 
150 XKM(II) = -1.0 
GO TO 200 
C 

C INITIAL XKM * 0 ••• 

160 XKM(II) = 0.0 
GO TO 230 
C 

C ••• DETERMINE INITIAL XKM > 0. ITERATION REQUIRED ••• 


170 CONTINUE 

XKM(II) = 1.0 
FG = 0.0 
1 80 CONTINUE 

XKM(II) = 2*XKM(II) 

WRITE (5,185) II,XKM(II),FG 

185 FORMAT (IX, 'ITERATE FOR XKM(IIs* ,13, • ) > 0.0. XKM =',F11.8, 
1 ' FG s' ,1PE16.8) 
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G = 1.0 

DO 190 IA=1 ,NIA 

G s G»(1 .0+XKM(II)»XK(II,IA)) 

190 CONTINUE 

FG = 1.0+XKM(II)-G 

IF (FG .GT. 0.0) GO TO 180 

GO TO 200 

C C 

C*«»C NEWTON-RAPHSON ITERATION FOR XKM. {MODULE 4} 

C 

200 CONTINUE 
C 

G = 1 .0 

DO 210 IA=1,NIA 

G s G*( 1 .0+XKM(II)«XK(II,IA)) 

210 CONTINUE 

FG = 1 .0+XKM(II)-G 
DG = 0.0 
DO 220 IAs1,NIA 

DG = DG+(XK( II, IA)/ ( 1 .0+XKM( II) # XK( II, IA) ) )*G 
220 CONTINUE 

DG s 1.0-DG 
XKML = XKM (II) 

XKM(II) = XKM( II)-FG/DG 
WRITE (5,225) II,XKM(II) 

225 FORMAT (IX, 'NEWTON-RAPHSON ITERATION. XKM(II=» ,13, ' ) s', F1 1.8) 
IF (DABS(XKM(II)-XKML) .GT. 1 .OE-8) GO TO 200 
GO TO 230 

C C 

C«*»C WRITE XKM(II) FOR INDIVIDUAL (II). {MODULE 5) 

C 

230 CONTINUE 
C 

WRITE (5,240) II,XKM(II) 

240 FORMAT (IX, 'MASTER SCALING CONSTANT FOR INDIVIDUAL (IIs',13, 


1 ») IS: ' ,F1 1 .8) 

C C 


C*««C WRITE XK(II,IA),SXK, AND XKM(II) TO DISK. {MODULE 6} 

C 

LF s Z'OA' 

C 

C WRITE ATTRIBUTE SCALING CONSTANTS XK(NII,NIA) 

WRITE (8,250) LF,LF,II 

250 FORMAT (A1/A1 , 'ATTRIBUTE SCALING CONSTANTS XK(II,IA) FOR ', 
1 'INDIVIDUAL (IIs',13,')') 

C 

DO 270 IAM=1,NIA,10 
IAN s IAM + 9 

IF (IAN .GE. NIA) IAN = NIA 

WRITE (8,260) LF, (XK(II,IA) , IA=IAM,IAN) 

260 FORMAT (A1 ,10F7. 4) 

IF (IAN .EQ. NIA) GO TO 280 
270 CONTINUE 
280 CONTINUE 
C 
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C WRITE SOM OF ATTRIBUTE SCALING CONSTANTS SXK ••• 

WRITE (8,290) LF,II,SXK 

290 FORMAT (A1 , ’SUM OF ATTRIBUTE SCALING CONSTANTS FOR INDIVIDUAL 
1 ' ( 11= 1 , 13 » ' ) IS:',F8.H) 

C 

C ••• WRITE MASTER SCALING CONSTANT XK(II) 

WRITE (8,300) LF,II,XKM(II) 

300 FORMAT (A1,‘ MASTER SCALING CONSTANT FOR INDIVIDUAL (II=* ,13,*)', 
1 » IS: *,F11.8) 

C C 

C««*C EXIT SUBROUTINE CALXKM. {MODULE 7} 

C 

WRITE (5,999) 

999 FORMAT (/' EXIT SUBROUTINE CALXKM') 

C 

RETURN 

C 

END 
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CHimfHHimiHHmmiHHmHmtmHiiimmmmifmiHc 

C SUBROUTINE CALUAT 2/7/83 

CfHHimiHiHHimmmiHiiimHHmmHHHiHiHHiiiimfc 

C SUBROUTINE CALUAT CALCULATES THE ATTRIBUTE UTILTITY FUNCION VALUES 
C UAT( IS, IP, IA) FOR THE ATTRIBUTES IA OF EACH OUTCOME (IS, IP) FOR EACH 
C INDIVIDUAL II. 

CiHimiHHmHimmumHmmiHHiimiHmHtHHHitiHtc 

SUBROUTINE CALUAT ( AT , UATD , U ATC , U AT , II , NIS , NIP , NIA , NID , NIC ) 

CimmHmiHHHIHHHHIHfHHtlHimHHIHIHHHIIIIHmiC 


C««*C INITIALIZE. {MODULE 1} 

DIMENSION 

1 AT(NIS,NIP,NIA) , 

2 UATD ( NIA, NID ) , UATC ( NIA , NIC ) , UAT( NIS , NIP , NIA) 

WRITE (5,100) 

100 FORMAT (/' ENTER SUBROUTINE CALUAT’) 

C C 

C«hh»C WRITE ATTRIBUTE UTILITY DATA FOR INDIVIDUAL (II). (MODULE 2} 
WRITE (5,110) II 

110 FORMAT (/IX, ’ATTRIBUTE UTILITY DATA UATD(IA.ID) FOR INDIVIDUAL’, 
1 ' ( 11 =' , 13 , ' ) ’) 

DO 11*0 IA=1 ,NIA 
WRITE (5,120) IA 

120 FORMAT (IX, ’ATTRIBUTE ’,13) 

WRITE (5,130) ( UATD ( IA, ID ) , ID= 1 , NID ) 

130 FORMAT (3( IX, 1 PE16 .** ,0PF7.H ) ) 

11*0 CONTINUE 

c c 

C»«»C CALCULATE ATTRIBUTE UTILITY FUNCTION COEFFICIENTS UATC(IA,IC) 

C FOR INDIVIDUAL (II). {MODULE 3) 

NICC = NIC - 1 
DO 170 IA=1 ,NIA 
DO 170 IC=1 , NICC, 2 
ATI = UATD(IA,IC) 

AT2 = UATD(IA, IC+2) 

UTIL1 = UATD(IA,IC+1) 

UTIL2 = UATD(IA,IC+3) 

IF (AT2 .NE. ATI) GO TO 150 
A = 0.0 
B = 0.0 
GO TO 160 
150 CONTINUE 

A = ( UTIL2 - UTIL1 )/(AT2 - ATI) 

B = UTIL1 - ATI *A 
160 CONTINUE 

UATC(IA,IC) = A 
UATC(IA, IC+1 ) = B 
170 CONTINUE 



C»»»C WRITE ATTRIBUTE UTILITY FUNCTION COEFFICIENTS UATC(IA,IC) 

C FOR INDIVIDUAL (II). {MODULE *»} 

WRITE (5,180) II 

180 FORMAT (/IX, 'ATTRIBUTE UTILITY FUNCTION COEFFICIENTS ’, 

1 ’UATC(IA,IC) FOR INDIVIDUAL (II=* ,13, * ) * ) 

DO 210 IA=1 , NIA 
WRITE (5,190) IA 
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190 FORMAT (IX, 'ATTRIBUTE *,I 3 ) 

WRITE (5,200) (UATC(IA,IC) ,IC =1 , NIC) 

200 FORMAT (5(1X,1PE14.4)) 

21 0 CONTINUE 

c c 

C»*«C WRITE ATTRIBUTE DATA AT(IS,IP,IA) FOR OUTCOMES (IS, IP). 

C {MODULE 5) 

WRITE (5,220) 

220 FORMAT (IX/ IX, 'ATTRIBUTE DATA AT(IS, IP, IA) ' ) 


DO 250 IS= 1 , NIS 


DO 250 IPs 1, NIP 

WRITE (5,230) IS, IP 

230 FORMAT (IX, 'ATTRIBUTE DATA FOR OUTCOME ', 

1 ' ( ISs ' , 13 , ' , IPs ' , 13 , ' ) ' ) 

WRITE (5,240) (AT(IS,IP,IA) ,IA=1 ,NIA) 

240 FORMAT (5( IX, 1 PE14 .4) ) 

250 CONTINUE 

C C 

C»*»C CALCULATE ATTRIBUTE UTILITIES UAT(IS,IP,IA) . {MODULE 6} 


DO 260 ISs 1, NIS 
DO 260 IPsI ,NIP 
DO 260 IAsI ,NIA 
DO 258 IDs3,NID,2 

IF ( (UATD( IA, 1 ) .LT. UATD(IA,3)) 


1 .AND. (AT(IS,IP,IA) .GT. UATD(IA,ID) ) ) 

2 GO TO 258 

IF ( (UATD(IA,1 ) .GT. UATD(IA,3)) 

1 .AND. (AT(IS,IP,IA) .LT. UATD(IA, ID) ) ) 

2 GO TO 258 

IC s ID - 2 

UAT(IS,IP,IA) s UATC(IA,IC)*AT(IS,IP,IA) 

1 + UATC(IA, IC+ 1 ) 

GO TO 260 

258 CONTINUE 

260 CONTINUE 

C C 

C««C WRITE ATTRIBUTE UTILITIES UAT(IS, IP,IA) FOR ALL OUTCOMES 
C (IS, IP). {MODULE 7) 

WRITE (5,265) II 

265 FORMAT (/IX, 'ATTRIBUTE UTILITIES FOR INDIVIDUAL (IIs • , 13 , • ) ' ) 

DO 290 ISs 1, NIS 
DO 290 IPs 1, NIP 

WRITE (5,270) IS, IP 

270 FORMAT ( IX, 'ATTRIBUTE UTILITIES UAT(IS,IP,IA) FOR ', 

1 'OUTCOME (ISs*, 13,', IPs', 13,’)') 

WRITE (5,280) (UAT(IS,IP,IA),IAs1 ,NIA) 

280 FORMAT (1X,10F7.4) 

290 CONTINUE 

C C 

C***C EXIT SUBROUTINE CALUAT. {MODULE 99} 

WRITE (5,999) 

999 FORMAT (/' EXIT SUBROUTINE CALUAT') 

C 

RETURN 
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C 

END 

CmiiKHHimHiiHiimiimimmmHHHHHitmiiiitmmic 
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CHiimimfitHHHmHHmmHHiiiHimimmitHtHHiftmc 

C SUBROUTINE CALUO 2/6/83 

C SUBROUTINE CALUO CALCULATES THE OUTCOME UTILITY FUNCTION VALUE 
C UO(IS,IF) FOR EACH OUTCOME (IS, IP) FOR EACH INDIVIDUAL (II). 

SUBROUTINE CALUO (XK,XK24, UAT, UO.II, Nil, NIS, NIP, NIA) 


C»»*C INITIALIZE. {MODULE 1} 

DOUBLE PRECISION XKM(NII), PROD 

DIMENSION XK(NII.NIA) ,UAT(NIS,NIP,NIA) ,UO(NIS,NIP) 

WRITE (5,100) 

100 FORMAT (/* ENTER SUBROUTINE CALUO') 


c c 

C«««C WRITE SUBROUTINE INPUT ARRAYS. {MODULE 2} 

WRITE (5,110) II 

110 FORMAT (/IX, 'ATTRIBUTE SCALING CONSTANTS FOR INDIVIDUAL (11=' , 

1 13,’)*) 

WRITE (5,120) (XK(II,IA),IA=1,NIA) 

120 FORMAT ( 10( 1X.F7.4) ) 

WRITE (5,130) II,XKM(II) 

130 FORMAT (IX, 'MASTER SCALING CONSTANT FOR INDIVIDUAL (11=' ,13, ' ) • , 
1 ' IS: ' ,F1 1 .8) 


WRITE (5,140) II 

140 FORMAT (/IX, 'ATTRIBUTE UTILITIES FOR INDIVIDUAL ( 11= ' , 13 , ' ) ' ) 
DO 170 IS=1,NIS 


DO 170 IP=1 ,NIP 

WRITE (5,150) IS, IP 

150 FORMAT ( 1 X ,' ATTRIBUTE UTILITIES UAT(IS,IP) FOR ', 

1 'OUTCOME (IS=',I3,’,IP=',I3,')') 

WRITE (5,160) (UAT(IS,IP,IA),IA=1 ,NIA) 

160 FORMAT (1X,10F7.4) 

170 CONTINUE 

C C 

C«««C TEST FOR ADDITIVE OR MULTIPLICATIVE UTILITY FUNCTION. 

C»«»C {MODULE 3} 

IF (XKM(II) .EQ. 0.0) GO TO 180 
IF (XKM(II) .NE. 0.0) GO TO 210 

C C 

C«»*C CALCULATE ADDITIVE UTILITIES U0(IS,IP) FOR OUTCOMES (IS, IP). 
C»»»c {MODULE 4} 

180 CONTINUE 


DO 200 IS* 1, NIS 
DO 200 IP= 1, NIP 
UO(IS,IP) =0.0 
DO 190 I As 1, NIA 

UO(IS,IP) = UO(IS.IP) + XK(II,IA)«UAT(IS,IP,IA) 


190 CONTINUE 

200 CONTINUE 
GO TO 235 

c c 


C«*«c CALCULATE MULTIPLICATIVE UTILITIES UO(IS,IP) FOR OUTCOMES 
C»»»C (IS, IP). {MODULE 5) 

210 CONTINUE 

DO 230 IS=1,NIS 


B-21 



File: B: CALUO. FOR 


DO 230 IP= 1, NIP 
PROD s 1.0 
DO 220 IA=1,NIA 

PROD s PROD*( 1 .0+XKM(II)*XK(II, IA)*UAT( IS, IP, IA) ) 

220 CONTINUE 

U0(IS,IP) = (PROD-1 .0)/XKM( II) 

230 CONTINUE 

C~— — — —C 
C«»»C WRITE UTILITIES U0(IS,IP) FOR OUTCOMES (IS, IP). {MODULE 6} 

235 CONTINUE 

WRITE (5,240) II 

240 FORMAT (/IX, ’OUTCOME UTILITIES (IS, IP) FOR INDIVIDUAL (IIs’,13, 

1 ’)’) 

DO 270 ISsI ,NIS 
WRITE (5,250) IS 

250 FORMAT (IX,’ OUTCOME UTILITIES (IS, IP) FOR SYSTEM (ISs • ,13, * ) ’ ) 
WRITE (5,260) (U0(IS,IP),IPs1 ,NIP) 

260 FORMAT (1X,10F7.4) 

270 CONTINUE 

C C 


C»#»c EXIT SUBROUTINE CALUO. {MODULE 99) 

WRITE (5,999) 

999 FORMAT (/* EXIT SUBROUTINE CALUO') 

C 

RETURN 

C 

END 

CHimmmimiiHHHmHmiHimiiHmiHimHmmimmc 
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CimimiHiiHHHHiiHHHHmiiimiiiiiiimmiHiiiimiiHic 

C SUBROUTINE CALUS 9/8/82 

CmHfllHHHHmHIHHHHIHHHimiHiHHHmtHIHimmiC 

C SUBROUTINE CALUS CALCULATES THE SYSTEM UTILITY FUNCTION VALUE 
C US(II,IS) FOR EACH SYSTEM (IS) FOR EACH INDIVIDUAL (II). 

CiHHHimHHHmmHHtmHiiiHiHimtimimmHimHmc 

SUBROUTINE CALUS( PROB, UO, US, R3, IRS, II, Nil, NIS, NIP) 


C»»»C INITIALIZE. {MODULE 1} 

C 

DIMENSION 

1 PROB (NIS, NIP ),U0( NIS, NIP), 

2 US( Nil , NIS ) , RS( Nil , NIS ) , IRS( Nil , NIS ) 

WRITE (5,100) 

100 FORMAT (/' ENTER SUBROUTINE CALUS') 

C C 

C«*«C WRITE SUBROUTINE INPUT ARRAYS. {MODULE 2} 

C 

WRITE (5,110) 

110 FORMAT (/IX, 'PROBABILITIES PR0B(IS,IP) FOR OUTCOMES (IS, IP)') 

DO 140 ISs 1 , NIS 
WRITE (5,120) IS 

120 FORMAT ( IX, 'PROBABILITIES PROB(IS.IP) FOR OUTCOMES (IS, IP) ', 

1 'FOR SYSTEM (ISs ’ , 13 , • ) ' ) 

WRITE (5,130) (PROB(IS,IP) ,IP=1 ,NIP) 

130 FORMAT (1X,10F7.4) 

140 CONTINUE 
C 

WRITE (5,150) II 


150 FORMAT (/IX, 'OUTCOME UTILITIES (IS, IP) FOR INDIVIDUAL (IIs',13, 

1 ')’) 

DO 180 ISs 1, NIS 
WRITE (5,160) IS 

160 FORMAT (IX, 'OUTCOME UTILITIES (IS, IP) FOR SYSTEM (ISs» ,13, • ) •) 
WRITE (5,170) (UO(IS,IP),IP=1 ,NIP) 

170 FORMAT (1X,10F7.4) 

180 CONTINUE 

C 


C*»«C CALCULATE UTILITIES US(II,IS) FOR SYSTEMS (IS). {MODULE 3) 

C 

DO 200 IS= 1, NIS 
US(II,IS) s 0.0 
DO 190 IPs 1, NIP 

US(II,IS) s US(II,IS) ♦ PR0B( IS, IP)*UO( IS, IP) 

190 CONTINUE 
200 CONTINUE 

C 

c«*»c WRITE UTILITIES US(II,IS) FOR SYSTEMS (IS) FOR INDIVIDUAL (II). 
C«»»C {MODULE 4} 

C 

WRITE (5,210) II 

210 FORMAT (/IX, 'UTILITIES US(II,IS) FOR INDIVIDUAL (IIs* , 13, ' ) •) 
WRITE (5,220) (US(II,IS) ,ISs1 ,NIS) 

220 FORMAT (1X,10F7.4) 

C 


C 
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C»«»C CALCULATE RANKS RS(II,IS) OF ALTERNATIVE SYSTEMS (IS). 
C«»»C {MODULE 5) 


DO 240 IS=1 ,NIS 
RS(II,IS) = 0.5 
DO 230 IIS=1 ,NIS 

IF (US(II,IS) .EQ. US(II,IIS)) RS(II,IS) = RS(II,IS) +0.5 
IF (US(II f IS) .LT. US(II,IIS)) RS(II,IS) = RS(II,IS) + 1.0 
230 CONTINUE 
240 CONTINUE 

C C 


C»»«C WRITE RANKS RS(II f IS) FOR SYSTEMS (IS) FOR INDIVIDUAL (II). 

C»**C {MODULE 6} 

C 

WRITE (5,250) II 

250 FORMAT (/IX, ‘SYSTEM RANKS RS(II,IS) FOR INDIVIDUAL (11= ‘,13,’)’) 
WRITE (5,260) (RS(II,IS) ,IS=1 ,NIS) 

260 FORMAT (1X,10F7.4) 

C C 

C«»«C CALCULATE INTEGER RANKS IRS(II,IS) OF ALTERNATIVE SYSTEMS (IS). 
C«««c {MODULE 7) 

C 

DO 280 IS=1 , NIS 
IRS(II, IS) = 1 
DO 270 IIS=1 , NIS 

IF (US(II,IS) .LT. US(II,IIS)) IRS(II f IS) = IRS(II,IS) + 1 
270 CONTINUE 
280 CONTINUE 

C C 

C»««C WRITE INTEGER RANKS IRS(II,IS) FOR SYSTEMS (IS) FOR INDIVIDUAL 
C«»«C (II). {MODULE 8} 

C 

WRITE (5,290) II 

290 FORMAT (/IX, ‘SYSTEM INTEGER RANKS IRS(II,IS) FOR INDIVIDUAL ’, 

1 '(II=‘ ,13, ’ ) ’ ) 

WRITE (5,300) ( IRS( II, IS) ,IS=1 , NIS) 

300 FORMAT (IX, 1017) 

C C 

C««C EXIT SUBROUTINE CALUS. {MODULE 9) 

C 

WRITE (5,999) 

999 FORMAT (/» EXIT SUBROUTINE CALUS») 

C 

RETURN 

C 

END 

CHHiiimHiiHmmfHimHmmmimiHHmiimiHHmmc 
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CHHtHHHiHmHHmmHmuiHHHimoHmimimtmiHic 

C SUBROUTINE GROUP 1/29/83 

CiiimfKHHmiiHtHiHmiimimimimomtHmiiitiHHHC 

C SUBROUTINE GROUP CALCULATES THE GROUP DECISION RULE VALUES VG(IG,IS) 
C AND THE RANKINGS RG(IG,IS) AND IRG(IG,IS) FOR THE SYSTEMS 
C (IS=1 ,...,NIS) FOR EACH GROUP DECISION RULE (IG=1,..,3). 

CiiHHHHmHmmHHHiHiiiiiHiiHmmimiiHHHiiiiiimc 

SUBROUTINE GROUP ( US , RS, IRS, VG, RG , IRG , Nil , NIS ) 


C*»»C INITIALIZE. {MODULE 1} 

C 

DIMENSION 

1 US(NII,NIS) ,RS(NII,NIS) ,IRS(NII,NIS) , 

2 VG(3,NIS),RG(3»NIS),IRG(3,NIS) 


C 

WRITE (5,100) 

100 FORMAT (/• ENTER SUBROUTINE GROUP') 

C C 

C«««C WRITE SUBROUTINE INPUT ARRAYS. {MODULE 2} 

C 

C WRITE UTILITIES US(II,IS) FOR SYSTEMS (IS) FOR INDIVIDUALS (II). 


WRITE (5,110) 

110 FORMAT (/IX, 'SYSTEM UTILITIES US(II,IS) FOR INDIVIDUALS (II)') 
DO U0 11=1 ,NII 


WRITE (5,120) II 

120 FORMAT ( IX, 'UTILITIES US(II,IS) FOR INDIVIDUAL (11= ' ,13, ' ) ’ ) 
WRITE (5,130) (US(II,IS) ,IS=1 ,NIS) 

130 FORMAT (1X.10F7.4) 

1U0 CONTINUE 
C 

C WRITE SYSTEM RANKS RS(II,IS) FOR INDIVIDUALS (II). 

WRITE (5,150) 

150 FORMAT (/IX, 'SYSTEM RANKS RS(II,IS) FOR INDIVIDUALS (II)') 

DO 180 11=1, Nil 
WRITE (5,160) II 

160 FORMAT (IX, 'SYSTEM RANKS RS(II,IS) FOR INDIVIDUAL (II*',I3, 

1 ’)') 

WRITE (5,170) (RS(II,IS) ,IS=1 ,NIS) 

170 FORMAT (1X,10F7.4) 

1 80 CONTINUE 

C 

C WRITE SYSTEM INTEGER RANKS IRS(II,IS) FOR INDIVIDUALS (II). 

WRITE (5,190) 

190 FORMAT (IX, 'SYSTEM INTEGER RANKS IRS(II,IS) FOR INDIVIDUALS ', 

1 ’(II)') 

DO 220 11=1, Nil 
WRITE (5,200) II 

200 FORMAT (IX, 'SYSTEM INTEGER RANKS IRS(II,IS) FOR INDIVIDUAL ', 

1 ' ( IIs ' , 13 » ' ) ' ) 

WRITE (5,210) (IRS(II,IS) ,IS=1 ,NIS) 

210 FORMAT (IX, 1017) 

220 CONTINUE 

C C 

C««C CALCULATION FOR ADDITIVE UTILITY RULE (IG = 1). {MODULE 3) 

C 
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C CALCULATE GROUP VALUES VG(1,IS). 

DO 2 HO IS= 1 , NIS 
SUM =0.0 
DO 230 11=1, Nil 

SUM = SUM + US(II,IS) 

230 CONTINUE 

VG( 1 , IS) = SUM/NII 
240 CONTINUE 
C 

C CALCULATE GROUP RANKS RG(1,IS). 

DO 260 IS=1,NIS 
RG( 1 ,IS) = 0.5 
DO 250 IIS =1 , NIS 

IF (ABS(VG(1 , IS) - VG( 1 , IIS) ) .LE. 0.5E-4) 

1 RG( 1 , IS) = RG( 1 , IS) + 0.5 

IF (ABS(VG( 1 , IS) - VG(1,IIS)) .LE. 0.5E-4) GO TO 250 
IF (VG( 1 , IS) .LT. VG(1,IIS)) RG(1,IS) = RG(1,IS) + 1.0 
250 CONTINUE 

260 CONTINUE 
C 

C CALCULATE GROUP INTEGER RANKS IRG(1,IS). 

DO 280 IS= 1 , NIS 
IRG(1,IS) = 1 
DO 270 IIS =1 ,NIS 

IF (ABS( VG( 1 , IS) - VG(1,IIS)) .LE. 0.5E-4) GO TO 270 
IF (VGd.IS) .LT. VG( 1 , IIS) ) IRG(1,IS) = IRG(1,IS) ♦ 1 
270 CONTINUE 

280 CONTINUE 

c 

C»«*C CALCULATION FOR NASH BARGAINING RULE (IG = 2). {MODULE 4} 

C 

C CALCULATE GROUP VALUES VG(2,IS). 

DO 300 IS=1 , NIS 
PROD = 1.0 
DO 290 11=1, Nil 

PROD = PROD # US( II , IS) 

290 CONTINUE 

PROD = ABS(PROD) 

VG(2,IS) = PROD**( 1 .0/NII) 

300 CONTINUE 
C 

C CALCULATE GROUP RANKS RG(2,IS). 

DO 320 IS= 1 , NIS 
RG(2,IS) = 0.5 
DO 310 IIS = 1 , NIS 

IF (ABS(VG(2,IS) - VG(2,IIS) ) .LE. 0.5E-4) 

1 RG(2,IS) = RG(2,IS) + 0.5 

IF (ABS(VG(2,IS) - VG(2,IIS) ) .LE. 0.5E-4) GO TO 310 
IF (VG(2,IS) .LT. VG(2,IIS) ) RG(2,IS) = RG(2,IS) + 1.0 
310 CONTINUE 
320 CONTINUE 
C 

C CALCULATE GROUP INTEGER RANKS IRG(2,IS). 

DO 340 IS= 1 , NIS 
IRG(2,IS) = 1 
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DO 330 IIS =1 ,NIS 

IF (ABS(VG(2,IS) - VG(2,IIS) ) .LE. 0.5E-4) GO TO 330 
IF (VG(2,IS) .LT. VG(2,IIS) ) IRG(2,IS) = IRG(2,IS) + 1 
330 CONTINUE 
340 CONTINUE 

C 

C««C CALCULATION FOR RANK SUM RULE (IG = 3). {MODULE 5} 

C 

C CALCULATE GROUP VALUES VG(3,IS) 

DO 350 IS=1,NIS 
VG(3»IS) = 0.0 
DO 350 11=1 ,NII 

VG(3,IS) = VG(3»IS) ♦ RS(II,IS) 

350 CONTINUE 

DO 360 IS= 1 , NIS 

VG(3»IS) = 1.0 - ((VG(3,IS) - NII)/(NIS*NII - Nil)) 

360 CONTINUE 
C 

C CALCULATE GROUP RANKS RG(3,IS). 

DO 380 IS=1 ,NIS 
RG(3,IS) = 0.5 
DO 370 IIS=1,NIS 

IF (ABS(VG(3,IS) - VG(3 »HS) ) .LE. 0.5E-4) 

1 RG(3,IS) = RG ( 3 » IS ) + 0.5 

IF (ABS( VG(3»IS) - VG(3 »HS) ) .LE. 0.5E-4) GO TO 370 
IF (VG(3,IS) .LT. VG(3,IIS)) RG(3,IS) = RG(3,IS) + 1.0 
370 CONTINUE 

380 CONTINUE 
C 

C CALCULATE GROUP INTEGER RANKS IRG(3,IS). 

DO 400 IS=1 ,NIS 
IRG(3»IS) = 1 
DO 390 IIS=1,NIS 

IF (ABS(VG(3,IS) - VG(3 »HS) ) .LE. 0.5E-4) GO TO 390 
IF (VG(3,IS) .LT. VG(3*HS) ) IRG(3,IS) = IRG(3,IS) + 1 
390 CONTINUE 

400 CONTINUE 

C 

C»««C WRITE GROUP CALCULATION RESULTS. {MODULE 7) 

C 

WRITE (5,H10) 

410 FORMAT (//IX, ’SYSTEM ADDITIVE RULE NASH RULE 

1 » RANK SUM RULE’/IX,’ (IS) VALUE RANK IRANK 

2 » VALUE RANK IRANK VALUE RANK IRANK ' / 

3 IX,’ '* 

Jl * ') 

DO 430 IS= 1 , NIS 

WRITE (5,420) IS,VG( 1 ,IS) ,RG( 1 ,IS) ,IRG( 1 ,IS) ,VG(2,IS) , 

1 RG(2,IS) ,IRG(2,IS) ,VG(3,IS) ,RG(3,IS) ,IRG(3,IS) 

420 FORMAT (1X,I3,3(F10.4 .FT.^.K) ) 

430 CONTINUE 

C 

C***C EXIT SUBROUTINE GROUP. {MODULE 8} 

C 

WRITE (5,999) 
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999 

C 

C 

C« 


FORMAT (/• EXIT SUBROUTINE GROUP*) 

RETURN 

END 


File: BiCONCRD.FOR 


CtHfHiHHitMmiimmmmiiiiHHfmifHHimmiHmimtc 

C SUBROUTINE CONCRD 7/21/83 

C SUBROUTINE CONCRD CALCULATES THE CONCORDANCE STATISTICS FOR 
C RANKINGS BY INDIVIDUALS OR BY GROUP DECISION RULES. 

SUBROUTINE CONCRD( RX , SRX , TIES , STIES , W , CHISQR , JDF , IW , NIIX , NIS) 

CmfmiHiHiHiitmHimHHHHifiHimHHimmiimHHHfC 


C»«*C INITIALIZE. {MODULE 1} 

C 

DIMENSION 

1 RX( NIIX, NIS) , 

2 SRX(NIS) ,TIES( NIIX, NIS) ,STIES(NIIX) , 

3 W(2),CHISQR(2),JDF(2) 

C 

WRITE (5,100) 

100 FORMAT (/» ENTER SUBROUTINE CONCRD’) 

C 

WRITE (5,110) IW, NIIX, NIS 

110 FORMAT (/IX, ’IW = ’ ,11 ,5X, ’NIIX = \I3,5X,’NIS = ’,13) 

C 

DO 140 11*1, NIIX 
WRITE (5,120) II 

120 FORMAT ( IX, ’ RX(II, IS) FOR II =',I3) 

WRITE (5,130) (RX(II,IS) ,IS=1 ,NIS) 

130 FORMAT (1X,5F10.4) 

140 CONTINUE 

C 

C«*«C CALCULATE SUMSQR s SUM OF SQUARES. {MODULE 2} 

C 

C «•* SUM OF RANKINGS BY SYSTEM (IS) •** 

DO 145 IS=1,NIS 
SRX (IS) =0.0 
1 45 CONTINUE 

DO 150 IS=1 ,NIS 
DO 150 11=1, NIIX 

SRX (IS) = SRX (IS) + RX(II,IS) 

150 CONTINUE 
C 

WRITE (5,155) 

155 FORMAT (/IX, ’SRX(IS) ’) 

WRITE (5,160) (SRX(IS) ,IS=1 ,NIS) 

160 FORMAT (1X.5F10.4) 

C 

C MEAN OF SUM OF RANKINGS •« 

SRXM =0.0 
DO 170 IS=1 ,NIS 

SRXM = SRXM + SRX(IS)/NIS 
170 CONTINUE 
C 

WRITE (5,180) SRXM 
180 FORMAT (/IX,’ SRXM = *,3X,F10.4) 

C 

C ••• SUM OF SQUARES ••• 

SUMSQR =0.0 
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DO 190 IS=1,NIS 

SUMSQR = SUMSQR + (SRX(IS) - SRXM)»»2 
190 CONTINUE 

WRITE (5,200) SUMSQR 
200 FORMAT (IX, ’SUMSQR = ’,F10.4) 

C 

c 

c»»«c CALCULATE TIES CORRECTION. {MODULE 3) 

C 

DO 204 11=1, NIIX 
DO 204 IS =1 ,NIS 
TIES(II, IS) =0.0 
204 CONTINUE 

DO 206 11=1 , NIIX 
STIES(II) =0.0 
206 CONTINUE 
C 

DO 260 11=1, NIIX 
DO 250 IS=1,NIS 
DO 220 IIS=1 , NIS 

IF (IIS .EQ. IS) GO TO 220 

IF ( (ABS( RX( II, IIS) - RX(II,IS) ) .LE. 0.5E-4) 

.AND. (IIS .LT. IS)) GO TO 250 

IF (TIES(II,IS) .GT. 0.0) GO TO 210 

IF ( ABS( RX(II, IIS) - RX(II, IS) ) .LE. 0.5E-4) 

TIES( II, IS) = 2.0 
GO TO 220 

CONTINUE 

IF (ABS(RX(II,IIS) - RX(II,IS) ) .LE. 0.5E-4) 

TIES( II, IS) = TIES(II,IS) + 1.0 

CONTINUE 

WRITE (5,230) II , IS , TIES( II , IS) 

FORMAT (IX, ’II = ' , I3,5X, ’IS = *,I3,10X, 

’TIES(II, IS) = ’ ,F10.4) 

STIES(II) = STIES(II) 

+ ( 1 .0/12.0)*(TIES(II,IS)**3 - TIES(II,IS)) 
WRITE (5,240) II,STIES(II) 

FORMAT (IX, 'II = ' , 13 , 5X , ’ STIES( II ) = ’,F10.4) 

250 CONTINUE 
260 CONTINUE 
C 

TTIES =0.0 
DO 270 11=1, NIIX 

TTIES = TTIES + STIES(II) 

270 CONTINUE 

WRITE (5,280) TTIES 
280 FORMAT (IX, 'TTIES = ’,F10.5) 


C 

1 

C 

C 

1 

C 

210 

1 

C 

220 

C 

230 

1 

C 

1 

240 
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c . c 

C««»C CALCULATE W(IW) = CONCORDANCE, JDF(IW) = DEGREES OF FREEDOM, AND 
C»«*C CHISQR(IW) s CHI SQUARE VALUE. {MODULE 4} 

C 

W(IW) = SUMSQR/ ((1.0/12.0)*( NIIX**2 ) * ( NIS**3 - NIS) - NIIX*TTIES) 
C 

JDF(IW) s NIS - 1 
C 

CHISQR(IW) = NIIX*(NIS - 1)»W(IW) 

C 

WRITE (5,290) IW,W(IW) ,IW,CHISQR(IW) ,IW, JDF(IW) 

290 FORMAT (1X,’W(\I1,’) s ’ ,F10.4 ,1 OX, ’CHISQR( * ,11 , 1 ) s »,F10.4, 


1 10X,’JDF(’,I1,») = »,I5) 

c c 


C«»c EXIT SUBROUTINE CONCRD. {MODULE 5) 

C 

WRITE (5,999) 

999 FORMAT (/' EXIT SUBROUTINE CONCRD’) 

C 

RETURN 

C 

END 

CiiiiifimiHiiimmiiKiKiHHHHfmfmtfmmmmimHiic 
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CHiHmHiimimmmmtHHHHitimmfmHiimmmiiiHc 

C SUBROUTINE OUTPUT 9/8/82 

C SUBROUTINE OUTPUT FORMATS AND OUTPUTS THE RESULTS OF THE 
C CALCULATIONS. 

cmHimiiimmiHmiHiHmiimiHiHHimmHHHHiimiic 

SUBROUTINE OUTPUT ( US, IRS, VG, IRG , W, CHISQR, JDF , Nil, NIS) 

CimHimHiiiHHimiiHiiimiiHiimiimmiHHimimomc 


C«»*C INITIALIZE. {MODULE 1} 

DIMENSION 

1 US(NII,NIS) , IRS( Nil, NIS) ,VG(3,NIS) ,IRG(3,NIS) , 

2 W(2) ,CHISQR(2) , JDF(2) 

WRITE (5,100) 

100 FORMAT (/» ENTER SUBROUTINE OUTPUT') 

C C 

C««C DO LOOP FOR OUTPUT UNIT. {MODULE 2} 

DO 280 IU=1 ,2 

IF (IU .EQ. 1) LF = Z'20' 

IF (IU .EQ. 1) JUNIT = 5 

IF (IU .EQ. 2) LF = Z'OA' 

IF (IU .EQ. 2) JUNIT = 8 

WRITE (5,110) JUNIT 
110 FORMAT (/IX, 'JUNIT s \I3/1X) 

C C 

C»«C OUTPUT PREFERENCE PAGES BY INDIVIDUALS (II). {MODULE 3) 

C 

ISM = 1 

ISN = 20 

IF (ISN .GT. NIS) ISN = NIS 

IIM = 1 

I IN = 5 

IF (IIN .GT. Nil) IIN s Nil 

C 

C PAGE HEADER ••• 

C 

120 CONTINUE 

WRITE (JUNIT, 130) LF,LF,LF,LF 

130 FORMAT (A1/A1,29X, 'MULTIATTRIBUTE DECISION ANALYSIS'/ 

1 A1.27X,’ RANKING OF ALTERNATIVE SYSTEMS (IS)'/ 

2 A1,35X,'BY INDIVIDUALS (II)') 

WRITE (JUNIT, 140) LF,LF, (II, 11= IIM, IIN) 

140 FORMAT (A1/A1 ,6X, ' IS' ,5X, «IIs ' ,13 ,4( 8X, '11= ' ,13) ) 

WRITE (JUNIT, 150) LF 
150 FORMAT (A1,8X,5(' UTILITY RANK’ ) ) 

C 

C *•* INDIVIDUAL PREFERENCES (US AND IRS) ••• 

C 

DO 170 IS=ISM, ISN 

WRITE (JUNIT, 160) LF,LF,IS, (US(II,IS) ,IRS(II,IS) ,II=IIM,IIN) 
160 FORMAT (A1/A1 ,5X,I3,2X,5(F6.4,2X,I3,3X)) 

170 CONTINUE 

C 

C ••• PAGE LOGIC ••• 

C 

IF (IIN .EQ. Nil .AND. ISN .EQ. NIS) GO TO 190 
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IF (IIN .EQ. Nil) GO TO 180 
IIM = IIM+5 
IIN = IIN+5 

IF (IIN .GT. Nil) IIN = Nil 
GO TO 120 
1 80 CONTINUE 

ISM = ISM +20 
ISN = ISN+20 

IF (ISN .GT. NIS) ISN = NIS 
IIM s 1 
IIN s 5 

IF (IIN .GT. Nil) IIN s Nil 
GO TO 120 


190 CONTINUE 

C 

C •** CONCORDANCE FOR INDIVIDUALS 

C 

WRITE (JUNIT.200) LF,LF,W(1 ) ,CHISQR( 1 ) , JDF( 1 ) 

200 FORMAT (A1/A1,'W r ' ,F6.4,10X, 'CHI SQUARE s ’,F10.2,10X, 

1 ’DF s' ,13) 



C***C OUTPUT PREFERENCE PAGES BY GROUP DECISION RULES (IG). 

C {MODULE 4} 


C 

ISM s 1 
ISN = 20 

IF (ISN .GT. NIS) ISN = NIS 
C 

C ••• PAGE HEADER ••• 

C 

210 CONTINUE 

WRITE ( JUNIT,220) LF,LF,LF,LF 

220 FORMAT (A1/A1 ,1 9X, 'MULTI ATTRIBUTE DECISION ANALYSIS’/ 

1 A1 ,17X, ’RANKING OF ALTERNATIVE SYSTEMS (IS)*/ 

2 A1 ,21X, 'BY GROUP DECISION RULES (IG)’) 

WRITE (JUNIT.230) LF,LF,LF,LF 

230 FORMAT (A1/A1 ,6X, ’IS' ,7X, 'IG s 1',10X,'IG = 2’,10X,'IG s 3'/ 

1 A1,14X, ’ADDITIVE’, 10X, ’NASH', 10X, ’RANK SUM’/ 

2 A1 ,7X,3(6X,» VALUE RANK’)) 

C 

C ••• GROUP PREFERENCES (VG AND RG) ••• 

C 

DO 250 IS=ISM,ISN 

WRITE (JUNIT,240) LF,LF,IS,(VG(IG,IS) ,IRG(IG,IS) ,IGs1 ,3) 
240 FORMAT (A1/A1 ,6X,I2,4X,3(F6.4,2X,I2,6X)) 

250 CONTINUE 

C 

C ••• PAGE LOGIC ••• 

C 

IF (ISN .EQ. NIS) GO TO 260 
ISM = ISM +20 
ISN = ISN+20 

IF (ISN .GT. NIS) ISN = NIS 
GO TO 210 
260 CONTINUE 
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C 

C CONCORDANCE FOR GROUPS 

C 

WRITE (JUNIT.270) LF,LF,W(2) ,CHISQR(2) , JDF(2) 

270 FORMAT (A1/A1,'W r * ,F6.4,10X, ’CHI SQUARE = ’ ,F10.2,10X, 

1 ’DF a' ,13) 

C C 

C»»*C END DO LOOP FOR JUNIT. {MODULE 5} 

C 

280 CONTINUE 

C — — — — — ------------- — -c 

C«»C EXIT SUBROUTINE OUTPUT. {MODULE 6} 

C 


WRITE (5,999) 

999 FORMAT (/’ EXIT SUBROUTINE OUTPUT’) 
C 

RETURN 

C 

END 



File: B : ARRAY . FOR 


£***********************:|^**********:|c********************************£ 

c ARRAY. FOR 6/25/83 

£#** »*****************************************************************£ 
C THIS ROUTINE IS TO BE "INCLUDED" IN THE MAIN ROUTINE AT COMPILE 
C TIME. IT DIMENSIONS ALL THE ARRAYS. 
£***#****************************************#**#*#******************£ 


C***C FORMAT FOR DIMENSIONING THE ARRAYS. ALL THE ARRAY DIMENSIONS 
C MUST BE REPLACED WITH NUMBERS. 

C 

C WHERE A "/" IS PRESENT, ENTER THE LARGER OF THE TWO VALUES ON 
C EITHER SIDE. 

C 

C DOUBLE PRECISION XKM(NII) 

C DIMENSION 

C 1 PROB(NIS.NIP) , AT(NIS,NIP,NAT) , 

C 2 UATD(NAT.NID) , UATC ( NAT , NIC ) , UAT( NIS , NIP , NAT ) , 

C 3 XK(NII ,NAT) , 

C 4 UO ( NIS , NIP ) , US(NII.NIS), RS(NII.NIS), 

C 5 VG(3 ,NIS) , RG( 3 , NIS ) , 

C 6 IRS (Nil, NIS) , IRG(3,NIS) , 

C 7 RX(NI 1 /3 , NI S ) , SRX(NIS ) , TIES (Nil /3 , NIS ) , STIES (NI 1/3) , 

C 8 W( 2 ) , CHISQR( 2 ) , JDF ( 2 ) 

C c 

C***C DIMENSION THESE ARRAYS WITH NUMBERS. 

C 

DOUBLE PRECISION XKM(IO) 

DIMENSION 

1 PROB( 18,1), AT( 18,1,10), 

2 UATD (10,10), UATC (10,8) , UAT( 18 , 1 , 10) , 

3 XK ( 5 , 10 ) , 

4 UO ( 18 , 1 ) , US ( 5 , 18 ) , RS ( 5 , 18 ) , 

5 VG ( 3 , 18 ) , RG ( 3 , 18 ) , 

6 IRS ( 5 , 18 ) , IRG( 3 , 18 ) , 

7 RX( 5 , 18 ) , SRX( 18 ) , TIES (5, 18) ,STIES(5) , 

8 W( 2 ) , CHISQR( 2 ) , JDF ( 2 ) 

q ***************************** ARRAY FOR ****************************£ 
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MATEUS RUN PARAMETERS 

J RUN ID JD1AG Nil NIS NIP 


53 


0 4 


16 


i 

PROBABILITY 

DATA 

FOR OUTCOMES 

( IS , IP) 

OF 

SYSTEM 

(IS=1) 

1 .000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

( IS=2 ) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

( IS, IP) 

OF 

SYSTEM 

(IS-3) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

(IS-4) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(is, IP) 

OF 

SYSTEM 

( IS=5) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

(IS-6) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

(IS=7) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

(IS-8) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

( I S-9 ) 

1 .000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

( IS-10) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

(IS=U) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

( IS=12) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

( I S = 1 3 ) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(is, IP) 

OF 

SYSTEM 

( I S=14 ) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

( I S = 1 5 ) 

1.000 







PROBABILITY 

DATA 

FOR OUTCOMES 

(IS, IP) 

OF 

SYSTEM 

( IS=16 ) 

1.000 







ATTRIBUTE DATA AT(IS,IP,IA) FOR OUTCOME 

(IS=1,IP 

= D 

7.0 


42.0 

8.0 



6.0 

7.0 


4.0 

3.0 




ATTRIBUTE DATA AT(IS,IP,IA) FOR OUTCOME 

( 1 S = 2 , IF 

'=1 ) 

7.0 


100.0 

6.0 



7.0 

6.0 


4.0 

3.0 




ATTRIBUTE DATA AT(IS,IP,IA) FOR OUTCOME 

( IS-3 , IF 

’=1) 

7.0 


31.0 

7.0 



7.8 

7.0 


4.0 

3.0 




ATTRIBUTE DATA AT(IS,IP,IA) FOR OUTCOME 

( IS=4 , IF 

‘=D 

7.0 


27.0 

4.0 



6.9 

5.0 


2.0 

3.0 




ATTRIBUTE DATA AT(IS,IP,1A) FOR OUTCOME 

(IS=5 f IF 

-1 ) 

7.0 


60.0 

4.0 



6.9 

6.0 


2.0 

3.0 





ATTRIBUTE DATA AT(IS,IP,IA) FOR OUTCOME (IS=6,IP=1) 

7.0 30. 0 9.0 7.2 

8.0 5.0 3.0 


NIA 

8 


Nin 

10 


193. 0E6 
198.0E6 
124. 0E6 
140. 0E6 
114. 0E6 
143. 0E6 
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ATTRIBUTE 

DATA 

AT ( I S , I P , I A ) 

FOR 

OUTCOME 

( IS=7 , IP=1 ) 


3.0 


50.0 


2.0 

3.8 

213. 0E6 

5.0 


9.0 


3.0 



ATTRIBUTE 

DATA 

AT ( I S , I P , I A ) 

FOR 

OUTCOME 

( IS=8 , IP= 3 ) 


8.0 


42.0 


8.0 

6.0 

200. 0E6 

8.0 


8.0 


3.0 



ATTRIBUTE 

DATA 

AT( IS , IP , IA) 

FOR 

OUTCOME 

( IS=9 , IP=1 ) 


8.0 


100.0 


7.0 

7.0 

190. 0E6 

7.0 


8.0 


3.0 



ATTRIBUTE 

DATA 

AT( IS f IP v IA) 

FOR 

OUTCOME 

(1S=10, IP=1) 


8.0 


31.0 


8.0 

7.8 

124. 0E6 

8.0 


3.0 


3.0 



ATTRIBUTE 

DATA 

AT(IS, IP, IA) 

FOR 

OUTCOME 

( IS=11 , IP=1) 


8.0 


27.0 


5 . 0 

6.9 

160. 0E6 

6.0 


4.0 


3.0 



ATTRIBUTE 

DATA 

AT ( IS , IP , IA) 

FOR 

OUTCOME 

( IS=12 , IP=3 ) 


8.0 


45.0 


5.0 

7.1 

114. 0E6 

7.0 


4.0 


3.0 



ATTRIBUTE 

DATA 

AT(IS, IP, IA) 

FOR 

OUTCOME 

( IS=13 , IP=1 ) 


8.0 


108.0 


5.0 

3.9 

240. 0E6 

5 . 0 


9.0 


3.0 



ATTRIBUTE 

DATA 

AT( IS , IP , IA) 

FOR 

OUTCOME 

( I S=14 , IP-3 ) 


8.0 


67.0 


10.0 

6.3 

135. 0E6 

10.0 


10.0 


3.0 



ATTRIBUTE 

DATA 

AT( IS, IP, IA) 

FOR 

OUTCOME 

( IS=15 , IP=1 ) 


8.0 


80.0 


10.0 

7.4 

135. 0E6 

10.0 


10.0 


3.0 



ATTRIBUTE 

DATA 

AT ( IS , IP , IA) 

FOR 

OUTCOME 

( IS--16 , IP=1 ) 


6.0 


38.0 


7.0 

7.6 

170. 0E6 

9.0 


10.0 


3.0 




SCALING CONSTANTS XK(II,IA) FOR INTERVIEW #5. II = i 
0.670 0.500 0.620 0.700 0.200 0.300 0.450 0.500 


ATTRIBUTE 

UTILITY DATA UATD(IA.ID) 

FOR INTERVIEW #5. 

11=1 

ATTRIBUTE 

1 





3.0 

0.00 

5 . 5 

0.25 

6.5 

0.50 

6.8 

0.75 

8.0 

1.00 



ATTRIBUTE 

2 





108.0 

0.00 

95.0 

0.25 

85.0 

0.50 

72.5 

0.75 

27.0 

1.00 



ATTRIBUTE 

3 





2.0 

0.00 

4.5 

0.25 

5.0 

0.50 

6.5 

0.75 

10.0 

1.00 



ATTRIBUTE 

4 





3.8 

0.00 

4.8 

0.25 

5 . 5 

0.50 

6.8 

0.75 

7.8 

1.00 



ATTRIBUTE 

5 

240. 0E6 0.00 


195. 0E6 0.25 


1.70. 0E6 0. 


140. 0E6 0.75 


114. 0E6 1.00 



ATTRIBUTE 

6 





5.0 

0.00 

5.8 

0.25 

6.5 

0.50 

7.3 

0.75 

10.0 

1 .00 




50 
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ATTRIBUTE 

7 





2.0 

0.00 

4.2 

0.25 

5 . 5 

0.50 

6.0 

0.75 

10.0 

1.00 



ATTRIBUTE 

8 





3.0 

0.00 

4.79 

0.25 

4.8 

0.50 

7.0 

0.75 

8.0 

1.00 



SCALING CONSTANTS XK(II,IA) 

FOR INTERVIEW #10. 1.1=2 


0.800 0. 

100 0.500 0.700 

0.100 

0.300 0.200 

0.600 


ATTRIBUTE 

UTILITY DATA UATD(IA.ID) 

FOR INTERVIEW 

#10. 11=2 


ATTRIBUTE 

1 





3.0 

0.00 

5.0 

0.25 

6.0 

0.50 

7.0 

0.75 

8.0 

1.00 



ATTRIBUTE 

2 





108.0 

0.00 

90.0 

0.25 

75.0 

0.50 

40.0 

0.75 

27.0 

1.00 



ATTRIBUTE 

3 





2.0 

0.00 

5 . 0 

0.25 

7.0 

0.50 

8.0 

0.75 

10.0 

1.00 



ATTRIBUTE 

4 





3.8 

0.00 

5.0 

0.25 

6.0 

0.50 

7.0 

0.75 

7.8 

1 . 00 



ATTRIBUTE 

5 






240. 0E6 0.00 


230. 0E6 0.25 

200. 

0E6 0 . 50 


180. 0E6 0.75 


114. 0E6 1.00 



ATTRIBUTE 

6 





5 . 0 

0 . 00 

5 . 5 

0 . 25 

6.0 

0.50 

7.0 

0.75 

10.0 

1.00 



ATTRIBUTE 

7 





2.0 

0.00 

3.0 

0.25 

4.0 

0.50 

5.0 

0.75 

10.0 

1.00 



ATTRIBUTE 

8 





3.0 

0.00 

5.0 

0.25 

6.0 

0.50 

7.0 

0.75 

8.0 

1.00 



SCALING CONSTANTS XK(il,IA) 

FOR IN 

TER VI EW #2. II 

=3 


0.600 0 

.100 0.500 0.400 

0.200 

0.200 0.400 

0.300 


ATTRIBUTE 

UTILITY DATA UATD(IA.ID) 

FOR INTERVIEW 

#2. 11=3 


ATTRIBUTE 

1 





3.0 

0.00 

3.5 

0.25 

5 . 5 

0.50 

6.0 

0.75 

8.0 

1.00 



ATTRIBUTE 

2 





108.0 

0.00 

85.0 

0.25 

67.0 

0 . 50 

47.0 

0.75 

27.0 

1.00 



ATTRIBUTE 

3 





2.0 

0.00 

4.0 

0 . 25 

6.0 

0 . 50 

8.0 

0.75 

10.0 

1 .00 



ATTRIBUTE 

4 





3.8 

0.00 

4.4 

0.25 

5 . 0 

0.50 

6.5 

0.75 

7.8 

1.00 




ATTRIBUTE 5 

240. 0E6 0.00 190. 0E6 0.25 175. 0E6 0.50 

145. 0E6 0.75 114. 0E6 1.00 
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ATTRIBUTE 

6 





5.0 


0.00 

6.7 

0.25 7.5 

0.50 

8.5 


0.75 

10.0 

1.00 


ATTRIBUTE 

7 





2.0 


0.00 

4.0 

0.25 6.0 

0.50 

8.0 


0.75 

10.0 

1.00 


ATTRIBUTE 

8 





3.0 


0.00 

4.2 

0.25 5.5 

0.50 

6.5 


0.75 

8.0 

1.00 


SCALING CONSTANTS XK(11,IA) 

FOR INTERVIEW #1. 11=4 


0.850 0. 

400 

0.550 0.450 

0.450 

0.550 0.250 0.350 


ATTRIBUTE 

UTILITY DATA UATD(IA.ID) 

FOR INTERVIEW #1 . 11-4 


ATTRIBUTE 

1 





3.0 


0.00 

3.01 

0.25 3.02 

0.50 

3.03 

0.75 

8.0 

1.00 


ATTRIBUTE 

2 





108.0 


0.00 

85.0 

0.25 70.0 

0.50 

60.0 


0.75 

27.0 

1 .00 


ATTRIBUTE 

3 





2.0 


0.00 

5.0 

0.25 6.0 

0.50 

7.0 


0.75 

10.0 

1.00 


ATTRIBUTE 

4 





3.8 


0.00 

6.0 

0.25 7.0 

0.50 

7.4 


0.75 

7.8 

1.00 


ATTRIBUTE 

5 






240. 

. 0E6 0.00 


180. 0E6 0.25 140, 

. 0E6 0 . J 


122 

. 5E6 0.75 


114. 0E6 1.00 


ATTRIBUTE 

6 





5 . 0 


0.00 

5.75 

0.25 6.5 

0.50 

9.0 


0.75 

10.0 

1.00 


ATTRIBUTE 

7 





2.0 


0.00 

4.0 

0.25 6.5 

0.50 

8.5 


0.75 

10.0 

1.00 


ATTRIBUTE 

8 





3.0 


0.00 

4.0 

0.25 6 . 5 

0.50 

6.75 

0.75 

8.0 

1.00 
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MATEUS 

MULTIATTRIBUTE EVALUATION OF UTILITIES 


RUN IDENTIFICATION NUMBER: 53 
DIAGNOSTIC DISPLAY LEVEL: 0 
NUMBER OF INDIVIDUALS: 4 
NUMBER OF SYSTEMS: 16 
NUMBER OF PROBABILITY PATHS: 1 
NUMBER OF ATTRIBUTES: 8 
NUMBER OF ATTRIBUTE DATA: 10 


ATTRIBUTE SCALING CONSTANTS XK(II.IB) FOR INDIVIDUAL (11= 1) 

.6700 .5000 .6200 .7000 .2000 .3000 .4500 .5000 

SUM OF ATTRIBUTE SCALING CONSTANTS FOR INDIVIDUAL (11= 1) TS: 3.9400 

MASTER SCALING CONSTANT FOR INDIVIDUAL (11= 1) IS: -.99702036 

ATTRIBUTE SCALING CONSTANTS XK(Il.IB) FOR INDIVIDUAL (1.1= 2) 

.8000 .1000 .5000 .7000 .1000 .3000 .2000 .6000 

SUM OF ATTRIBUTE SCALING CONSTANTS FOR INDIVIDUAL (11= 2) IS: 3.3000 

MASTER SCALING CONSTANT FOR INDIVIDUAL (11= 2) IS: -.99424555 

ATTRIBUTE SCALING CONSTANTS XK(I1 , IB) FOR INDIVIDUAL (11= 3) 

.6000 .1000 .5000 .4000 .2000 .2000 .4000 .3000 

SUM OF ATTRIBUTE SCALING CONSTANTS FOR INDIVIDUAL (11= 3) IS: 2.7000 

MASTER SCALING CONSTANT FOR INDIVIDUAL (11= 3) IS: -.96579052 

ATTRIBUTE SCALING CONSTANTS XK(JI.IB) FOR INDIVIDUAL (11= 4) 

.8500 .4000 .5500 .4500 .4500 .5500 .2500 .3500 

SUM OF ATTRIBUTE SCALING CONSTANTS FOR INDIVIDUAL (II - 4) IS: 3.8500 

MASTER SCALING CONSTANT FOR INDIVIDUAL (11= 4) IS: -.99722735 


MULTIATTRIBUTE DECISION ANALYSIS 
RANKING Ob' ALTERNATIVE SYSTEMS (IS) 
BY INDIVIDUALS (II) 


IS 

11 = 

UTILITY 

1 

RANK 

11 = 

UTILITY 

2 

RANK 

11 = 

UTILITY 

3 

RANK 

11 = 

UTILITY 

4 

RANK 

1 

.9549 

11 

.9045 

J 1 

. 806 l 

11 

.9664 

10 

2 

.9153 

14 

.8924 

12 

.8178 

12 

.9278 

14 

3 

.9807 

5 

.9526 

7 

.8976 

8 

.9856 

4 

4 

.9219 

12 

.8546 

15 

.8016 

15 

.9376 

13 

5 

.9216 

13 

.8730 

13 

.8096 

13 

.9586 

12 

6 

.9726 

7 

.9501 

8 

.9150 

6 

.9748 

8 

7 

.6848 

16 

.2747 

16 

.4138 

16 

.4969 

16 

8 

.9812 

4 

.9601 

5 

.9213 

5 

.9793 

7 


11 = 

UTILITY RANK 
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9 

.9709 

9 

.9621 

4 

.9074 

7 

.9675 

9 

10 

.9939 

1 

.9859 

2 

. 9559 

3 

.9927 

2 

11 

.9650 

10 

.9492 

9 

.8662 

10 

.9630 

11 

12 

.9719 

8 

.9577 

6 

.8860 

9 

.9795 

6 

13 

.8720 

15 

.8617 

14 

.8032 

14 

.8989 

15 

14 

.9893 

3 

.9758 

3 

.9710 

2 

. 9913 

3 

15 

.9922 

2 

.9860 

1 

.9771 

1 

.9928 

1 

16 

.9779 

6 

.9276 

10 

.9268 

4 

.9822 

5 

9408 


CHI 

SQUARE = 


56 . 45 


DF = 15 



MULTI ATTRIBUTE DECISION ANALYSIS 
RANKING OF ALTERNATIVE SYSTEMS (IS) 
BY GROUP DECISION RULES (IG) 


IS IG = 1 IG = 2 IG = 3 

ADDITIVE NASH RANK SUM 



VALUE 

RANK 

VALUE 

RANK 

VALUE 

RANK 

1 

.9205 

11 

.9194 

11 

. 3500 

11 

2 

.8883 

13 

.8873 

13 

.2000 

13 

3 

.9541 

5 

. 9535 

5 

.6667 

5 

4 

.8789 

14 

.8772 

14 

.1500 

14 

5 

.8907 

12 

.8889 

12 

.2167 

12 

6 

.9531 

7 

.9528 

7 

.5833 

7 

7 

.4675 

16 

.4435 

16 

.0000 

16 

8 

.9605 

4 

.9602 

4 

.7167 

4 

9 

.9520 

8 

.9516 

8 

. 5833 

7 

10 

.9821 

2 

.9820 

2 

.9333 

2 

11 

.9359 

10 

.9350 

10 

.4000 

10 

12 

.9488 

9 

.9480 

9 

.5833 

7 

13 

.8590 

15 

.8582 

15 

.1000 

15 

14 

.9819 

3 

.9818 

3 

.8833 

3 
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15 

.9870 

1 

.9870 

1 

.9833 

1 

1 6 

.9537 

6 

. 9533 

6 

.6500 

6 

9987 


CHI SQUARE = 

44.94 


DF 
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